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I . INTRODUCTION 



A. OVERVIEW 

Top management personnel often attempt to quantify or 
measure the value of information systems technology to their 
organization. Managers within the Information Systems (IS) 
community, in turn, debate and search for improved techniques 
to measure and, otherwise, make improvements to both existing 
and future systems. 

In 1984 and 1986, the Society for Information Management 
(SIM) along with the Management Information Systems Research 
Center (MISRC) conducted surveys to determine key information 
systems issues facing IS executives/managers over the next 
five years. An issue ranked among the top ten in importance 
and one which has long been an IS management problem was 
system effectiveness and its measurement ( Brancheau and 
Wetherbe, 1987). 

Inasmuch as the Department Of Defense (DoD) has a number 
of large, expensive information systems being used at various 
Medical Treatment Facilities, a simple and efficient method to 
measure and analyze the effectiveness of these systems is 
essential. This is a very difficult concept to narrow down 
because of differing opinions or ideas of value (ie; economic. 
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efficiency, effectiveness, user satisfaction, etc.). Many 
scholars of this problem have determined and agreed that all 
of these factors are important ones, however, there is a need 
to concentrate on the effectiveness of a system in terms of 
user interface. In other words, user satisfaction of the 
system. This is due to the fact that no matter how "good" the 
system may be, if the user is not satisfied with it, it 
becomes worthless. However, if the user is satisfied, the 
system may be considered to be effective, provided all other 
organizational objectives are met. 



B . RESEARCH OBJECTIVES 

This study is a follow-on to the Evaluation of User 
Information Satisfaction of the Composite Health Care System 
(CHCS) (Hurd, 1991) and will empirically evaluate the 
effectiveness of the Automated Quality of Care Evaluation 
Support System (AQCESS) and compare them to those found for 
CHCS. A standardized, previously validated, satisfaction 
survey was administered to users of AQCESS at the Naval 
Hospital, Pensacola, Florida and Silas B. Hayes Hospital, Fort 
Ord, California to measure intrinsic satisfaction 
characteristics. The results of the survey will be analyzed 
using relevant statistical methods in order to identify and 
document problem areas if they exist, as well as those areas 
considered to be positive aspects of the system. Upon 
completion, this study will formulate a baseline measure of 
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the AQCESS user's satisfaction at both sites and investigate 
possible cause and effect relationships. Future comparisons 
can be made using the same standard survey at the study 
hospital or at other facilities. 
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II. THE AUTOMATED QUALITY OF CARE EVALUATION SUPPORT SYSTEM 



B . BACKGROUND 

The Tri-Service Medical Information Systems (TRIMIS) 
Program Office was formally created in July 1974, by the 
Department of Defense Assistant Secretaries of Defense 
(Comptroller, Health and Environment) . The program, which is 
now part of Defense Medical Systems Support Center (DMSSC) of 
the Office of the Assistant Secretary of Defense Health 
Affairs, was assigned the task of consolidating previous 
service efforts and to "improve the effectiveness and economy 
of health care delivery in the Army, Navy and Air Force." 
TRIMIS refined its mission objectives by stating that it would 
develop automated information systems for timely patient- 
centered health data. These systems would support medical 
services, clinical research, epidemiological studies and 
health care information. 

The Hospital Services Program Office (HSPO) developed a 
computer supported clinical records and patient administration 
system using the MUMPS language combined with certain 
utilities from the Veterans Administration File Manager. The 
system received extensive tri-service input and was designed 
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incorporated comprehensive service-specific information to 
ensure accurate and reliable data accumulation. The system 
was also designed to be user-friendly (easy to learn, able to 
provide online assistance, and able to operate without 
dedicated computer operators/special environmental 
conditions). The MUMPS language allowed for Hardware 
flexibility (easily modified to correct problems and integrate 
additional requirements). Development of this system, 
however, was suspended following redirection of the TRIMIS 
program in March of 1984. (AQCESS User Manual, Issues 2.0, 
2.2, 2.4) 

B. AQCESS PURPOSE 

In August of 1984, the Health Affairs Office of the 
Assistant Secretary of Defense identified medical quality 
assurance as being a priority within the military health care 
system. The TRIMIS Office, along with representatives from 
the Army, Navy and Air Force, were requested by the 
Professional Affairs and Quality Assurance Office of the 
Deputy Assistant Secretary of Defense to develop and implement 
a computer supported information system at all DoD hospitals 
by 1985. The resulting system was the Automated Quality of 
Care Evaluation Support System (AQCESS). This system, also 
written in MUMPS, was designed to satisfy the following 
objectives: (1) Collect and report clinical, administrative 
and managerial information necessary to support inpatient 
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administration of the medical quality assurance programs 
within DoD; (2) Improve the quality and timeliness of health 
care evaluation; and (3) Support the identification of 
variations that would adversely affect the quality of health 
care. (Assistant Secretary of Defense, Health Management 
Systems, 1984) 

C. ACCESS SYSTEM MODULES 

AQCESS is an interactive, mini-computer based, online 
computer system. It includes the following modules: 

1. Admission and Disposition 

Admission and Disposition addresses patient 
registration, admission, transfer, disposition, bed 
management, inpatient history and reporting. The user inputs 
patient information/data onto a formatted screen and has the 
capability to manipulate (ie; edit, view, cancel, track, 
print, validate) as necessary. From there, a number of other 
items may be generated (ie; register number, remarks section, 
inpatient history, query of patient database, general/specific 
reports ) . 

2. Clinical Records 

Clinical Records includes documentation on patient 
episodes, diagnosis and procedure data capture, patient day 
computation, record tracking and reporting. The user can 
collect, edit, validate, display, track various items, 
compute/maintain data and generate reports. 
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3 . Quality Assurance 

Quality Assurance supports inpatient and emergency 
room occurrence screening, problem tracking, offers solutions/ 
recommendations, provider profiling, credentialling, incident 
reporting, problem audit tracking, drug utilization, infection 
control, blood utilization review and reporting, risk 
management and surgical case review. Additionally, through 
monitoring and evaluation, the QA subsystem can gather data 
pertaining to the MTF's administrative procedures. It is then 
used to generate reports and trend analysis with respect to 
the quality of care. 

4 . Ad Hoc Reporting 

Ad Hoc Reporting enables users to produce AQCESS 
reports in support of management and clinical research 
studies. It can produce very detailed reports, extract data 
from multiple files, compute new data from existing data, 
enable any data item to be ported to any deliverable form, use 
boolean logic selection, conduct multi-level sorting, perform 
computations, provide flexible report formatting, generate 
MUMPS code, support system security. 

5. Embosser Interface 

Embosser Interface supports a number of patient 
(input) information card embossers. The embosser produces a 
credit card-sized, raised letter card which is subsequently 
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used much like a credit card (patient information obtained 
from card for input into the system). 

6. Interface to the Defense Enrollment Eligibility 

Reporting System (DEERS) 

Interface to DEERS provides the capability to check 
patient eligibility at the time of admission as well as input 
changes/corrections to both the AQCESS and DEERS databases. 

7. Business Office 

Business Office includes cashier processing and a 
number of other accounting and general business/office 
functions. It enables users to calculate inpatient/outpatient 
bills, review schedule of accounts, post accounts received, 
display and update the rate schedule summary and issue 
receipts/invoices . 

8. Outpatient Encounter 

Outpatient Encounter supports outpatient clinics and 
provides them with the capability to schedule, cancel or scan 
for appointments (patient appointments that are pending and to 
search for available slots), process a waiting list request, 
create/edit a schedule, check in patients for appointments and 
enter registration data for new patients. 

9 . Emergency Room 

Emergency Room allows users to record patient 
emergency room visits; produces an automated SF Form 558 
( Emergency Treatment Record ) ; produces the automated emergency 
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room log; and stores, online, a history of the encounter. The 
Emergency Room module is no longer used because of reduced 
administrative manpower ( system seems to be too time consuming 
under ER conditions). 

D . OPERATION 

AQCESS utilizes a centralized database and displays 
data/information by means of a series of screens. It provides 
real-time information processing and generates several 
automated outputs on a daily and monthly basis. The terminal 
displays a sign-on screen on which the user enters a user ID 
and password. After the system validates this input, the 
AQCESS main menu is displayed. (If the user enters an invalid 
ID and/or pass-word more than a predetermined number of times, 
"User Entry" will lock the terminal and user ID. ) The menu 
indicates the functions authorized to that user along with the 
specific functions which can be performed at that particular 
terminal. From the main menu, AQCESS data/information is 
subsequently displayed through a series of approximately 100 
screens associated with each sub-menu/ selection. The number 
of different responses users can make to each of the displays 
of data entered for each patient record (must consider a 
number of unique combinations) can be estimated as being 
infinite. There are a number of recovery and error correction 
procedures. However, software errors which cause the user to 
be deleted from the system are only corrected by the Systems 
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corrected by the Systems Manager or technical assistants. If 
a user is authorized to access data via a systems function, in 
most cases he/she is also authorized to update the data 
available through that function at any time. An error will 
result when a user (e.g.,in admissions) attempts to update 
certain types of data elements (e.g.,the source of admission) 
because these data elements can only be updated/changed by the 
Systems/Corrections Manager. 

As patients are registered and admitted, the user inputs 
demographic data and the DEERS eligibility check is performed. 
An embossed card is issued, if requested, and it contains 
input information ( register/account number, patient name, 
family member prefix, sponsor SSN, date of birth, sex) for 
access to patient's record. Upon patient's release from the 
MTF, further data is entered by the dispositions clerk, 
cashiering clerk and clinical records office. The database 
can then provide screen or hardcopy reports on current 
listings of patients admitted and discharged, current 
inpatients, patients on convalescent leave, patients 
subsisting elsewhere, patients on medical hold as well as 
listings of inpatient or delinquent bills. Once the patient 
has been admitted, the data is also tracked by the Quality 
Assurance Coordinator until he/she is discharged. 
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1 . System Mamagement 

System Management allows the System Manager to modify 
system tables (which define the valid entries for specific 
data fields) , maintain hospital profile data, allows manager 
the flexibility to decide whether to assign register numbers 
automatically or manually, generate user ID's and passwords, 
assign functional privilege to users/terminals, broadcast 
system announcements, disable and re-enable user logon 
capability system-wide, control the active hours for the DEERS 
line, view the report queue and interrupt processing, archive 
records, perform system back-ups, list software error logs, 
load software updates and monitor disk space usage. 

2 . Security 

Because of the sensitive nature of the data collected 
and reported by AQCESS, security of this data is of utmost 
importance. Briefly, system security is ensured by the 
following measures: 

• Use of systems functions and access to data displays and 
reports are restricted to authorized users and designated 
terminals, as well as password, 

• Data is included on reports in the form of specific codes 
to protect confidentiality. 

• The system includes a time-out feature, causing the screen 
display to disappear if a terminal is left unattended past 
a predetermined time period. 

• A privacy act statement is printed as the banner/cover 
page of every system report. (AQCESS User Manual, Issues 
2 . 0 , 2 . 2 , 2 . 4 ) 
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E. SYSTEM BENEFITS 



Generally, the overall benefits of AQCESS include; 

1 . automated data collection 

2. real-time processing 

3. easily accessible clinical information 

4. manipulation/analysis of data for easy generation of 
reports 

5. improved quality assurance monitoring 

6 . improved security features 

7. improved system management capabilities 
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III. DESCRIPTION OF THE COMPOSITE HEALTH CARE SYSTEM 



A . BACKGROUND 

The Department of Defense (DoD) has pursued the goal of 
providing automated computer support to its hospitals and 
clinics since 1968. In February 1979, the Mission Elements 
Need Statement (MENS), establishing the need for an automated 
computer system was approved. Under the direction of the Tri- 
Service Medical Information System (TRIMIS) program office, 
stand-alone and integrated health care computer systems were 
acquired, implemented and operated to support Pharmacy 
(TRIPHARM), Laboratory (TRILAB), Radiology (TRIRAD), 
Appointment and Scheduling (TRIPASS), Quality Assurance 
( AQCESS ) , and Hospital Information System (HIS). The 
knowledge and experience gained through the operation of these 
systems since 1979 was used to refine and validate the system 
requirements for a completely new and fully integrated 
information system the Composite Health Care System (CHCS). 

B . CHCS PURPOSE 

The primary purpose of CHCS is "to provide health care 
services that support military forces in fulfilling their 
required mission...." (Draft System Decision Paper, 1989). 
Improving the efficiency and effectiveness of military health 
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care delivery during peacetime and during mobilization through 
integration of information resources is the goal of CHCS. The 
major focus of CHCS is the integration of shared information 
resources to resolve the deficiencies of communication, 
decision support, and information processing. The Composite 
Health Care System is designed as a fully integrated medical 
information system that provides automated support of 
information requirements for military medical treatment 
facilities. CHCS supports the administrative functions and 
the delivery of health care with information retrieval 
services . 

C . DEPLOYMENT STATUS 

CHCS is undergoing the Operational Test and Evaluation 
(OT&E) phase and is currently deployed to Naval Hospital 
Charleston as one beta-test site. The role of a beta-test 
site is to incrementally replace the TRIMIS systems with 
validated CHCS software. Continued validation of requirements 
and extensive evaluation of the CHCS software is conducted 
prior to the decision to deploy CHCS worldwide. 

D . OPERATIONAL FEATURES 

CHCS supports various functional areas in sharing 
information. Each functional area uses this information for 
its own purpose and communicates results/activities to other 
areas. Functional areas supported by CHCS include: 
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1. Patient Administration 

Patient administration does the registration of a 
patient into the system. Once in the computer system, the 
patient's demographic information is available for access by 
all other modules. The patient administration module also 
performs the functions to admit patients to the hospital, 
transfer patients between wards, and discharge patients from 
the hospital. The creation, update and closing of inpatient 
records is also performed by the patient administration 
module . 

2. Patient Appointment and Scheduling 

The Patient Appointment and Scheduling module provides 
for a centralized appointment service, a decentralized 
appointment service, or a combination of both depending on the 
desire of the hospital. The appointment service creates and 
maintains the appointment schedule for the hospital. A 
patient is given an appointment date and time to see the 
health care provider creating a centralized repository of 
information to produce operational reports of activity. 

3. Nursing 

CHCS supports inpatient nursing in a variety of areas. 
In the area of patient care, nursing uses the system for 
entering and obtaining the status of all physician orders, 
entering patient assessment data, and generating patient care 
plans. Documentation of patient progress is performed in the 



15 



automated nursing notes. Nursing unit management uses CHCS to 
generate shift care plans, patient management reports, drug 
administration time reports, and staffing requirement reports. 
Routine administrative functions are performed by CHCS for 
nursing including; staff credentialling, documentation of 
continuing education and in-service training, and staff 
scheduling capabilities. 

4 . Laboratory 

The laboratory uses the CHCS system for processing of 
orders for laboratory tests with automated reporting of test 
results. Specimens are processed, tracked, and reported 
through the system. All results are placed into the patients 
automated medical record and are immediately available for 
inquiry by health care providers. Automated control of blood 
bank operations including; blood acquisition, inventory and 
utilization are accomplished through CHCS. The laboratory is 
supported with ability to manage inventory control, register 
patients into the tumor registry, conduct a drug testing 
program, and generate a multitude of management reports. 

5 . Pharmacy 

The pharmacy uses CHCS to process prescriptions for 
patients that are ordered by health care providers. The 
health care provider enters the prescription information at a 
terminal located in his office. The information is 
electronically transmitted to the pharmacy. The pharmacy can 
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immediately prepare the medication for dispensing before the 
patient leaves the physician's office. CHCS automatically 
performs checks for patient allergies, drug interactions with 
medications the patient is currently taking, drug dosage for 
the age of the patient, and records the prescription into the 
patient's medical record. In-patient order processing is 
accomplished in much the same fashion. On-line drug 
monographs are available for inquiry by medical staff at any 
terminal. Pharmacy management of inventory control and 
reporting are also accomplished. 

6. Radiology 

Orders for radiological procedures are electronically 
transmitted to the radiology department where the order is 
processed. Order tracking and results reporting are performed 
by the system. Radiology results are immediately available 
for inquiry by medical staff. Department management including 
inventory control is also conducted through CHCS. 

7. Clinical Dietetics 

The Clinical Dietetics service uses CHCS to obtain 
patient data needed to access the nutritional needs of the 
patient. This information is used to create patient diet 
plans, select patients of interest, patient menu selections 
and monitor patient nutritional data. CHCS is also used to 
maintain the nutrition clinic schedule and compute clinical 
dietetics workload data. 
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E. 



POTENTIAL BENEFITS 



In general terms, the overall benefits of the CHCS system 
include: 

1. reliable, timely, easily accessible clinical 
information 

2. improved documentation for medicolegal issues 

3. improved communication 

4. improved patient satisfaction 

5. improved management efficiency 
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IV. USER INFORMATION SATISFACTION AND ITS MEASUREMENT 



A. INTRODUCTION 

While the cost of hardware and telecommunications 
continues to decrease, the cost of software development and 
maintenance for "systems" is continuously rising. 
Organizations spend considerable amounts of time and money to 
create and implement useful/successful information systems. 
The first "hurdle" to be cleared is whether or not the system 
is really needed. If the need is there and the system is 
developed, the system's functionality must be evaluated (Is it 
doing what it's supposed to do?). Therefore, in addition to 
gauging the system by cost-benefit analysis, the effectiveness 
and intangible benefits of the system need to be evaluated. 

B. METHODS TO ASSESS BENEFITS/EFFECTIVENESS 

Several procedures have been suggested for assessing the 
benefits or effectiveness of an information system. Possibly 
due to the level of knowledge possessed by today's users, 
measuring user satisfaction is a common consideration in these 
procedures. User satisfaction is a very critical yardstick in 
measuring computer system success and failure. (Powers and 
Dickson, 1973) How can it be measured? 
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System usage (Swanson, 191 A', Conrath and Mignen, 1990) 
assumes that the effectiveness of an information system can be 
evaluated by the amount of time the system is used. Simply, 
one uses an effective information system more than a less 
effective system. The assumption here is that the use of the 
system is optional to the user. If, however, the use of the 
system is mandatory, the correlation between use and 
effectiveness can no longer be made. 

Others (e.g., Hamilton and Chervany, 1981) contend that 
effectiveness is determined by comparing performance to pre- 
determined objectives. To assess the effectiveness of an 
information system, the task objective of the system is first 
agreed upon. Standards are developed to measure how well the 
information system performs each task. Some of the problems 
which emerge include: objectives and measures are often not 
sufficiently defined; easily implemented, efficiency-oriented 
measures are regularly used over complex, effectiveness- 
oriented measures; individual or conflicting interpretations 
of what the objectives and corresponding data measures are, 
often exist. 

Another frequently used procedure to determine the 
benefits/value of an information system is economic evaluation 
(cost-benefit analysis). The benefits of a system are weighed 
against the cost for development and operation of the system 
(Nolan, 1974). Although this appears objective and 
comprehensive on paper, in practice, cost-benefit analysis is 
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highly subjective and difficult to conduct (Nolan, 1974). The 
difficulties in using this approach for research, as argued by 
Ives et al. (1983), results from: (1) many costs and benefits 
are intangible and not easily recognized and/or converted into 
monetary equivalents; (2) unstructured, ad hoc decision making 
benefits are nearly impossible to objectively assess; (3) even 
when these items are determined by an organization, the data 
are generally unrecorded and not available for research ( Ives 
et al . , 1983 ) . 

It has been theorized that evaluation of the effectiveness 
or success of an information system is best accomplished 
through the use of decision analysis. This is based upon 
studies which conclude that a user will interact with a system 
in order to obtain assistance in the decision making process 
if the system is perceived to be useful in that respect. In 
other words, there will be resulting productivity benefits. 
(Nolan, 1974) An effective information system supports a user 
by retrieving necessary information in the proper format, 
yields the required level of detail, displays it at the proper 
frequency and lends assistance to the decision-making process. 
However, the decision analysis approach suffers a significant 
disadvantage for use in scientific research in that it is non- 
quantifiable and cannot be easily replicated. 
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C. THE USER INFORMATION SATISFACTION CONSTRUCT 

The American College Encyclopedia Dictionary defines 
satisfaction as "the fulfillment of desires, expectations, 
needs or demands". Organizational Psychologist Bernard Bass 
(1965) generically defines satisfaction to mean the 

extent to which the item is rewarding to us relative to how 
much better we might do elsewhere and what aspirations we 
have " . 

A number of researchers have concluded that measuring User 
Information Satisfaction (UIS) is a viable substitute for 
measuring system success/effectiveness. Of note, however, are 
two empirical studies, (Aldag and Power, 1986; Gallupe and 
DeSanctis, 1988). These studies were conducted to find a 
relationship between satisfaction and actual system 
performance using objective third party judges. The results 
of these studies are conflicting and do not support the 
assumption that increased satisfaction indicates actual system 
performance increases. Although results have been mixed, 
researchers continue to diligently work towards validating and 
standardizing UIS measures. MIS practitioners and researchers 
agree that the user is the best gauge for determining the 
success or effectiveness of an information system. (Neumann 
and Segev, 1980; Ives et al., 1983; Bailey and Pearson, 1983; 
Baroudi and Orlikowski, 1988). 

The originators of the concept of UIS were Cyert and March 
(1963). Their research asserts that organizational behavior 
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continually imposes upon the manager the need for information. 
If a formal information system exits, the success or failure 
of that information system to meet the needs of the user 
either reinforces or frustrates the user's sense of 
satisfaction with that system. 

A user, who presumes that required information should be 
readily available through careful manipulation of an 
information system, carries out the necessary steps to 
retrieve that information. If the information is indeed 
readily available, then satisfaction with the system is 
reinforced. However, if the information is not readily 
available, the user must modify the search beyond the 
information system, possibly by some other less familiar 
procedure. As a result, the user becomes frustrated. 
Moreover, an elaborate exploration through an information 
system involves additional time and effort, hence, a decrease 
in productivity. From the user’s standpoint, information 
systems may actually hinder the user in carrying out the act 
of decision making (Nolan, 1965). 

Successive use by the user affords him/her the opportunity 
to continually evaluate the system. If over a period of time, 
the user discovers that the information system cannot retrieve 
specific information without the frequent use of ad hoc means, 
normally a response of user dissatisfaction with the 
information system occurs. The project is a failure if the 
end product does not satisfy those it was designed to serve 
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(Powers and Dickson, 1973). If, however, the user reliably 
obtains the required/ requested information through normal 
operation of the information system, satisfaction with the 
system will be reinforced and the information system is 
regarded as a success . 

D. THE MEASUREMENT OF USER INFORMATION SATISFACTION 

User Satisfaction, as a pervasive measure of system 
effectiveness, has gained wide acceptance as well as many 
improvements along the way. (Ives and Olson, 1979; Igbaria and 
Nachman, 1990). Neumann and Segev (1980) establish a 
correlation between a user’s reaction to factors having to do 
with satisfaction and his/ her perception of their 
organization’s performance. Swanson (1974) empirically found 
a high correlation between the user’s appreciation of the 
system and his/her usage of its outputs. Powers and Dickson 
(1973) concluded that user satisfaction is the most critical 
factor for system success. Albeit there is no standard 
measure of satisfaction in these studies, it is maintained 
that user information satisfaction is an indicator of system 
usage and success. 

The level of user information satisfaction in the studies 
previously referred to is derived from a multitude of factors. 
Many of the users were asked to evaluate computer services in 
terms of their sense of satisfaction (Bailey and Pearson, 
1983). The factors measured in the various studies included: 
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accuracy, content, frequency, timeliness, reliability, 
assistance, adequacy, accommodation, communication, access, 
appreciation and flexibility. Each of these studies used a 
measure that was unique to that particular study (Ives et al., 
1983), nevertheless, taken as a whole, provide insight into a 
description of UIS. 

Validation of the UIS measures in these studies is 
limited, especially in handling threats to internal validity 
(Haga and Zviran, 1990). Bailey and Pearson's (1983) model 
and questionnaire for measuring and analyzing UIS seemed to 
escape this deficiency. It was based on previous studies and 
provides the basis of an instrument, later refined (Ives et 
al., 1983), and validated ( Baroudi and Orlikowski, 1988) for 
use. Although the method employed to measure user 

satisfaction by this tool is a pre-experimental design lacking 
pre-test/post-test or some type of control group (Campbell and 
Stanley, 1966), Conrath and Mignen (1990) assert that it 
represents a key contribution in the development of a standard 
instrument to measure user satisfaction. 

Bailey and Pearson ( 1983 ) described a model proposed by 
Lawler and Wanous (1972) for measuring user satisfaction: 

i 

Si-Y. 

j=i 

where ; 

Rjj = The reaction to factor j by the individual i 

Wjj = The importance of factor j to individual i 
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Using this model, the satisfaction of the user is measured 
as the weighted sum of the user's positive and negative 
reactions against a set of information system factors (Bailey 
and Pearson, 1983). The user's perception of a "generically 
good" information system is described as one composed of 
factors which have previously been deemed to be most important 
and a system with which the user is very satisfied. 

To implement the model, a review of 22 studies dealing 
with computer-user interface was used to establish a set of 
factors representing the domain of user satisfaction. (Bailey 
and Pearson, 1983). The initial 36 factors generated from the 
literature review were expanded to 39 after further review by 
middle managers. They deduced that the 39 factors decided 
upon represent the domain of user satisfaction at a=0.01. 
Bailey and Pearson (1983) used four bipolar adjective pairs on 
a seven interval scale in order for users to explain and 
measure their perception of each factor. (the semantic 
differential technique; see Figure 4.1) The seven intervals, 
denoted by adverbial qualifiers, incrementally ranged from 
negative to positive feelings or vice versa. In addition to 

the four adjective pairs, the user was required to test 

internal consistency and/or assign importance to each factor 
via two additional scales. 

Format of Output; The material design of the layout and 

display of the output contents. 

good : : : ; : : : : bad 

simple : : : : ; : : : complex 
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readable :::::::: unreadable 



useful : : 

To me, this factor is 


• 


1 


• 


1 




: useless 


important : ; 


: 


1 


: 


: 


: 


: unimportant 



Figure 4.1 Illustration of questionnaire form 



Reliability coefficients of the Bailey and Pearson user 
information satisfaction questionnaire were determined to be 
very high, thus it was a reliable, valid instrument (Bailey 
and Pearson, 1983). It represented an important first step 
toward the development of a valid and useful UIS measure ( Ives 
et al. , 1983) . 

Deese ( 1979 ) used the Pearson questionnaire at the Federal 
Computer Performance Evaluation and Simulation Center. He 
professed, "The results identified problems that would not 
otherwise have been discovered" and maintained that the user 
satisfaction questionnaire was a very useful and worthwhile 
tool (Deese, 1979). 

Ives, Olson and Baroudi (1983) chose to undertake an 
in-depth assessment of the Pearson (1983) questionnaire. By 
replicating the initial Bailey and Pearson study, they too, 
were able to reinforce the validity and reliability of the 
instrument. They presented several approaches to improve the 
quality of the original Pearson instrument. 

The goal of the suggested improvements was to establish a 
standardized "short form" instrument. Since Pearson found 
that the importance scale provided no additional information. 
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it was eliminated from the instrument. Scales which displayed 
undesirable psychometric qualities were also eliminated. The 
number of items scaled within a question was reduced from four 
to two in order to reduce the time to complete the 
questionnaire. All of the scales in the original Pearson 
instrument were scored positively to the left and negatively 
to the right end of the scale. In order to reduce undesired 
outcome produced when a user simply marks down a column of 
responses, some of the scales were reverse scored, thereby 
increasing the reliability of the measure. The resulting 
short form was determined to substantially measure the concept 
of UIS that was originally proposed in the Pearson full 
instrument (Ives et al . , 1983). 

The short form measure (Ives et al., 1983) was further 
subjected to testing by Baroudi and Orlikowski (1988) to 
examine its psychometric properties. The final short form 
questionnaire consisted of 13 questions with two items 
evaluated per question (see Appendix A). Each item is scored 
on a seven interval scale, ranging from -3 (dissatisfied) to 
+3 (satisfied) with zero indicating a neutral response. 

The total individual user information satisfaction score 
is calculated by averaging the responses of the 2 items for 
each question and summing the scores of the 13 questions. 




Ri+Rj 

T~ 



28 



where : 

Sj = total individual user satisfaction 
= response to the first item scale 

R 2 = response to the second item scale 

The range of total satisfaction can be from -39 to +39 . 

Three factors: electronic data processing (EDP) staff and 
services (Factor A), information product (Factor B), and 
knowledge and involvement (Factor C) were found to comprise 
user satisfaction using factor analysis by Ives, Olson and 
Baroudi (1983). These subtotals are calculated as the average 
of the responses to questions loading into a particular 
factor. Questions 1, 2, 6, 11, and 12 load heavily into the 
electronic data processing (EDP) staff and services factor. 
Questions 7, 8, 9, 10 and 13 load heavily into the information 
product factor. Questions 2, 3, 4, and 5 load heavily into 
the knowledge and involvement factor. The factor subtotals 
will range from -3 to +3 in value. Averages are used for 
meaningful comparison between the three factor scores. 

Data gathered for the study came from 358 employees, 
mostly clerical and support personnel, of 26 New York area 
organizations. Construct validity was determined first by 
examining the relationship between each scale and the total 
UIS score. Factor analysis using varimax rotation converged 
to a three factor solution accounting for 68% of the variation 
in 5 iterations. This provides strong evidence of construct 
validity (Baroudi and Orlikowski, 1988). 



29 



Convergent validity was substantiated by comparing an 
interview measure of satisfaction with the instrument measure 
in two groups. One group of users were generally pleased and 
satisfied with the information system and the other group of 
users were generally dissatisfied. Administration of the 
instrument indicated a statistically significant difference by 
t-test in the groups at p < .001 (Baroudi and Orlikowski, 
1988) . 

The total satisfaction and subtotal scores reported a 
reliability level above the .80 required for research. This 
confirms the instrument to be internally consistent and 
reasonably free from measurement error. 

The work by Baroudi and Orlikowski (1988) and resulting 
short- form UIS instrument provides a reliable and valid 
measure of user information satisfaction. It furnishes a 
meaningful, standardized measure of overall satisfaction with 
an information system as well as specific information about 
satisfaction within the sub-factors of electronic data 
processing (EDP) staff and services, information product, and 
user knowledge and involvement. 
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V. 



RESEARCH METHODOLOGY 



A. INTRODUCTION 

The purpose of this research is to identify the 
characteristics of user information satisfaction (UIS) for 
users of the AQCESS system and compare the findings to those 
obtained for CHCS (Hurd, 1991). The study is based on 
statistical analysis of empirical data collected from AQCESS 
users at the Naval Hospital, Pensacola, FL and Silas B. Hays 
Hospital, Fort Ord, CA. The following sections will discuss 
the survey instrument, data collection methodology and methods 
of statistical analysis utilized in the study. 

B. THE SURVEY INSTRUMENT 

Avoiding the obstacles associated with developing a 
comprehensive survey questionnaire, the previously developed, 
psychometrically evaluated and validated short- form 
questionnaire of Baroudi and Orlikowski (1988) was used 
without alteration. A copy of the complete survey 
questionnaire is included in Appendix A. The questionnaire is 
composed of two sections: a demographical section and a user 
information satisfaction section. 
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1 . Demographic Section 



The first section of the questionnaire requested 
general information. Regarding questions 1-5, the respondents 
were asked to give their hospital division/department, job 
description, highest level of education, age and gender, 
respectively. To determine the user's experience level with 
the system, the sixth question asked for the length of time, 
in months, the respondent had used AQCESS. Further 
determination of the user's experience was sought in questions 
seven and eight. Question seven asked if the user had 
previously used other computer systems. If the respondent 
answered "yes" to question seven, he/she was asked if the 
previous system was a health care/hospital information system. 

2 . User Information Satisfaction Section 

The second part of the survey directly addressed the 
issues of user information satisfaction. The instrument was 
designed to obtain the user ' s opinion on how well the AQCESS 
system functioned. It consisted of thirteen questions; each 
question having two bipolar, adjective scales for responses. 
Each item was to marked with an "x" in one of seven values, 
ranging from -3 (extremely dissatisfied) to +3 (extremely 
satisfied), with zero indicating a neutral response. 

The questions asked can be categorized into one of 
three factors of user satisfaction. 
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Management Information Department /Information 

Management Division (MID/IMD) Staff; This factor is the 
respondent's perception of the attitude and responsiveness of 
the MID/IMD staff as well as their relationship with said 
staff. The MID/IMD staff provides only local support and act 
as a liaison to the Defense Medical Systems Support Center 
(DMSSC) which manages and administers the AQCESS program. 
Thus, this factor also takes into account services provided by 
DMSSC. 

Information Product: This factor is the respondent's 
perception of the quality of output delivered by AQCESS. 

Knowledge and Involvement: This factor is the 
respondent's perception of the quality of training provided, 
their understanding of the system, and their participation in 
its development /modif ications . 

C. SAMPLE AND DATA COLLECTION 

Naval Hospital, Pensacola and Silas B. Hays Hospital, Fort 
Ord are two of a multitude of U.S. Navy, Army, Air Force and 
Coast Guard sites which operate AQCESS. The sample population 
then consisted of personnel from in/out-patient, 
clinical/technical and administration areas. 

Local points of contact acted as distribution and 
collection agents for the surveys. The questionnaires, each 
accompanied with a cover letter (Appendix B), were delivered 
to a point of contact at each site. The points of contact 
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distributed 500 questionnaires at Naval Hospital, Pensacola 
and 200 at Silas B. Hays Hospital, Fort Ord (with return 
envelopes and instructions) to all department personnel within 
the hospital operating the AQCESS modules. The surveys were 
completed and returned by the respondents to the points of 
contact in sealed envelopes and were then forwarded to the 
researchers for analysis. 

There were 214 questionnaires returned from Pensacola and 
71 from Fort Ord, of which 174 and 58 surveys, respectively, 
had complete information. This represents a final return rate 
of 34.8% and 29%, respectively. 

D. STATISTICAL ANALYSIS METHODS 

Data from completed surveys were coded and entered into a 
spreadsheet program. The spreadsheet program was used to 
perform the statistical analysis. The specific procedures 
used in the analysis will be examined next. 

1. Respondent's Satisfaction Index 

Each respondent's total satisfaction index was 
calculated for the 13 questions using the following formula; 

^3 R ^R 



where : 

Si = Satisfaction Index for respondent i 
Raij = Response to first item scale of question j for 
respondent i 
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^bij ~ Response to second item scale of question j for 
respondent i 

2. Total Satisfaction Index 

The overall or total satisfaction index for the survey 
group is calculated by averaging the respondent's satisfaction 
index to find the mean. 

3. Question Averages 

The mean response to each of the thirteen questions is 
calculated for comparison of individual questions. 

4. Factor Subtotals 

The thirteen questions can be grouped into three 
factor subtotals, as defined by Ives, Olson, and Baroudi 
(1983); MID/IMD staff and services (Factor A); Information 
product (Factor B); and Knowledge and involvement (Factor C). 
The mean of the individual questions' averages is calculated 
to find the factor subtotals. Questions 1, 2, 6, 11, and 12 
are used for the MID/IMD staff and services subtotal. 
Questions 7, 8, 9, 10, and 13 are used for the information 

product subtotal. Questions 2, 3, 4, and 5 are used for the 
knowledge and involvement subtotal . 

5. Comparison of Groups 

The respondents were categorized by the type of use of 
the AQCESS system. Three work groups were identified: (1) 

Physicians; (2) Ancillary; and (3) Administration. The 
individual questions' averages, total satisfaction indexes, 
and sub- factor totals were compared for one group verses the 



35 



combination of the other two groups using a t-test with the 
significance level of alpha = 0.05. The t-test is used to 
determine if there is a significant difference between the 
arithmetic mean value of two groups. 
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VI. DESCRIPTIVE FINDINGS (NAVAL HOSPITAL, PENSACOLA) 



A. DEMOGRAPHIC FINDINGS 

Of the 174 respondents, 76 were male and 98 were female. 
The age of the respondents ranged from 19 to 77 with a mean of 
36 years old. 

1 . Work Center Distribution 

The hospital department work centers reported were 
in/out-patient areas including: administration (ie; 
Appointment and Scheduling, Medical Service Accounting, 
Emergency Room and Quality Improvement), clinics (ie; Physical 
Therapy, OB/GYN, Pediatrics, etc.), laboratory (ie; Radiology, 
Pathology, etc.) and other departments. Figure 6.1 reflects 
the distribution by work center. 
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2. Job Description Distribution 

The job descriptions reported are from a multitude of 
specific functions but can be categorized as: 

1. Technician - a hospital corpsman functioning in a 
medical-technical capacity (pharmacy technician, 
laboratory technician, clinic corpsman) 

2. Physician - a medical doctor 

3. Health Professional - a licensed medical 
professional other than a physician (nurse, 
pharmacist, physical therapist, bio-medical 
officer) 

4. Other - a person not listed above (hospital 
administration non-medical clinic staff) 

Figure 6.2 reflects the sample distribution by job 
description . 

PENSACOLA NAUAL HOSPITAL 

JOB DESCRIPTION 

<22 > 

TECHNICIfW <^7) 

PROFESSIONAL <20 > 



Figure 6 . 2 
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3. Work Group Distribution 

For the purposes of this investigation, the study 
population was segregated according to the assumed primary use 
of the computer system. The categories provide a clear 
separation of work groups from an organizational and cultural 
standpoint. These groups are used extensively throughout the 
study . 

1. Administrative - use of the system for indirect 
medical purposes (patient registration, 
appointment scheduling and administrative 
reporting ) 

2. Ancillary - use of the system for direct medical 
support (prescription filling, laboratory specimen 
processing ) 

3. Physician - use of the system by physicians 
(accessing patient historical data, performance 
reports ) 

Figure 6.3 represents the distribution of respondents 
according to their main use of the system. 

4. Education Distribution 

The level of education reported by the respondents 
represents a well educated user population. A little more 
than 79 percent of the users have had at least some college 
education with nearly 31 percent having completed a Bachelor, 
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Master, Medical or PhD degree. Figure 6.4 represents the 
breakdown by education. 



PENSACOLA NAUAL HOSPITAL 

WORK GROUP 



ANCILLrtOY (67 > 



PHYClSIflN (22) 




ftOtllNISTCATlUE (85> 



Figure 6.3 
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B. COMPUTER SYSTEM USE 



The length of time the respondents reported as having used 
the AQCESS computer system ranged from one to 72 months with 
a mean of 21 months. Of the 174 respondents, 63 had used a 
computer system previously. 58 of the 63 users had used a 
health care information system before using AQCESS. Although 
many of the respondents had experience with computers, this 
was the first exposure to a health care information system for 
67 percent of the users. 

C. SATISFACTION FINDINGS 

The survey results for user satisfaction are compared for 
the three work groups in three areas; 1) overall satisfaction, 
2) responses to individual questions, 3) three group factors; 
MID staff and services; information product; knowledge and 
involvement. The actual values of responses to individual 
questions reported in the survey may be significant in 
themselves. However, the relative scores are useful for 
comparison among various work groups, organizations and 
information systems. ( Baroudi and Orlikowski, 1988) The 
administrative group composes nearly 48 percent of the survey 
population, the ancillary group composes 40 percent and the 
remainder consists of physicians. 
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1. Overall Satisfaction (Pensacola) 

The overall satisfaction was determined by the mean of 
the sum of the 174 survey responses to the thirteen survey 
questions from the work groups at Pensacola. The mean overall 
satisfaction index of the combined work groups was 11.16 on a 
scale ranging from -39 to +39. Figure 6.5 illustrates a 
comparison of the overall satisfaction index of the work 
groups at Pensacola. The survey shows that the administrative 
group displays the highest overall satisfaction (13.81), 
closely followed by ancillary (13.73), with physicians 
registering the least overall satisfaction (-4.16). Using a 
t-test, the level of satisfaction for physicians was 
statistically significant lower at alpha =0.05 than remaining 
groups . 
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2. Individual Questions 

The 13 questions on the survey requires two responses 
from the respondent. A number of the response scales are 
reverse scored to nullify the effects of a respondent who 
simply marks down one column of the questionnaire (Baroudi and 
Orlikowski, 1988). Figure 6.6 displays the average level of 
satisfaction for each question, by work group, for Pensacola. 




It is easily discernable that some questions exhibit a 
relatively high or low level of satisfaction when compared to 
the other questions. Questions 1, 6 and 11 indicate a high 
level of user satisfaction. These questions deal with issues 
concerning the Management Information Department (MID) staff. 
However, questions 2, 3 and 12 stand out as exhibiting a 
relatively low satisfaction index. Questions 2 and 12 are 
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concerned with software development/modif ication and question 
3, training. The following discussion will examine each 
question by work group. 

Figure 6.7 illustrates Pensacola physician's responses to 
individual questions as they compare to the remainder of the 
survey group. Figure 6.8 represents the ancillary group's 
responses as compared to the remainder of the survey groups. 
Figure 6.9 is the administrative group's responses as compared 
to the remainder of the survey groups. 



Individual Question Response 
PHYSICANS vs OTHERS 




Question Number 



^ PhysWon ^ OTHERS 



Figure 6.7 



a. Qvestlon 1 - Relationship with the Management 
Information Department (MID) Staff 
This question measures the relationship between 
users and the Management Information Department. The level of 
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satisfaction towards the MID staff is relatively high. It has 
the highest satisfaction index. The administrative group 
showed the highest level of satisfaction of the three groups. 
The ancillary group was slightly lower. The physicians 
revealed a statistically significant lower difference at alpha 
= 0 . 05 . 



Individual Question Response 
ANCILLARY vs OTHERS 




Question Number 



I ANQaARY ^ OTHERS 



Figure 6.8 



Individual Question Response 
ADMINISTRATIVE vs OTHERS 




Question Number 



ADMNSTRATIVE ^ OTHERS 



Figure 6 . 9 
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b. Question 2 - Processing of Requests for Changes 
to Existing Systems 

This question is concerned with the length of time 
required and/or timeliness of changes to AQCESS. This was the 
lowest scoring question which indicates a user perceived 
problem. Physicians were the least satisfied with a 
statistically significant lower difference at alpha = 0.05. 
The administrative users displayed a statistically significant 
higher difference at alpha = 0.05. 

c. Question 3 - Degree of Training Provided to Users 
The response to this question indicates the user's 

perceived level of training to use the system. The level of 
satisfaction of the administrative and ancillary groups was 
nearly equal. The physicians, however, showed a statistically 
significant lower difference at alpha = 0.05. 

d. Question 4 - User’s Understanding of the System 
The response to this question is the respondent * s 

self-reported assessment of his/her ability to interact with 
the system. Physicians were statistically significant lower 
at alpha = 0.05. Whereas, the administrative group showed a 
statistically significant higher difference at alpha = 0.05. 
The ancillary group indicated they were substantially less 
satisfied with the amount of training they received (question 
2) as compared to their understanding of the system. 
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e. Question 5 - User's Feeling of Participation 

The responses to this question yield a measurement 
of the user's perception of participation in the system. The 
administrative group had a statistically significant higher 
perception of participation than the other groups. 
Conversely, physicians had a statistically significant lower 
difference at alpha = 0.05. The ancillary group's feeling of 
participation was slightly lower than the administrative, but 
not statistically significant. 

f. Question 6 - Attitude of the Management 
Information Department (MID) Staff 

The intent of this question is to gauge the user's 
perception of the attitude of the MID staff. In particular, 
the staff's attitude towards rendering services. The response 
to this question was positive for all three groups. The 
administrative group was statistically significant higher 
while the physicians were statistically significant lower at 
alpha = 0.05. However, this question ranked third among the 
physician's positive responses. 

g. Question 7 - Reliability of Output Information 
The user's response to this question indicates 

his/her view of consistency and dependability of the output 
information. Administrative and ancillary groups responses 
were positive and approximately the same. Physicians' 
response was statistically significant lower at alpha = 0.05. 
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h. Question 8 - Relevancy of Output Information ( to 
intended function) 

This question measures the degree of compatibility 
between what the user wants or requires and what is provided 
by the information products and services. Administrative 
group showed a statistically significant higher difference at 
alpha = 0.05. The physicians, however, displayed a 
statistically significant lower difference at alpha = 0.05. 
The ancillary group was slightly lower than the administrative 
group with no statistical significance. 

i. Question 9 - Accuracy of Output Information 

The user ' s opinion of correctness of the output 
information is captured by this question. Administrative 
group was statistically significant higher while physicians 
were statistically significant lower. Ancillary was not 
statistically significant, but was equally satisfied as the 
administrative group. 

j. Question 10 - Precision of Output Information 
Precision of output information is the variability 

of output information from that which it purports to measure. 
The administrative and ancillary groups were not statistically 
significant, but the administrative group's response was 
approximately twice that of the ancillary group's. The 
physicians had a statistically significant lower difference at 
alpha = 0.05. 
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k. Question 11 - Communication with the Management 

Information Department (MID) Staff 

The response to this question is a basis for 
determining user perception of the manner and methods of 
information exchange between users and the MID staff. The 
results for the administrative and ancillary groups were not 
statistically significant, but instead, the scores were nearly 
even. Physicians did, however, show a statistically 

significant lower difference at alpha = 0.05. 

l. Question 12 - Time Re<iuired for New Systems 

Development 

This question is designed to assess the user's 
perception of the elapsed time between his/her requests for 
new applications and the design, development and the 
implementation of the application systems by the contractor or 
outside service representative. The administrative and 
ancillary groups were nearly neutral. The physicians, however 
displayed a statistically significant lower difference at 
alpha = 0.05. 

m. Question 13 - Completeness of Output 

The response to this question was intended to help 
measure the user's impression of the comprehensiveness of the 
content of the output information. The administrative group 
were statistically significant difference higher, while the 
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physicians were statistically significantly lower at alpha = 
0.05. 

3 . Grouped Factors 

The original user’s information satisfaction survey 
instrument sub-divided the thirteen questions into three 
grouped factors. These factors are designated as EDP 
(MID/IMD) staff and services, information system product, and 
knowledge and involvement. (Ives, Olson and Baroudi, 1988 ) The 
three factors are calculated from the averages of the 
component questions. MID staff and services (Factor A) is 
sub-divided into two separate components: local MID staff 
functions (MID) and contractor or Defense Medical Systems 
Support Center (DMSSC) functions (Con). Factor A (MID) is 
derived from questions 1, 6, and 11; Factor A (Con) from 
questions 2 and 12; information system product (Factor B) from 
questions 7, 8, 9, 10 and 13; and knowledge and involvement 
(Factor C) from questions 2, 3, 4, and 5 (Hurd, 1991). 

The administrative group's and physicians' level of 
satisfaction were statistically different at significance 
level alpha = 0.05. The administrative group was the most 
satisfied and the physicians were the least satisfied of all 
the groups. Figure 6.10 represents the work groups' level of 
satisfaction with relation to sub-total factors. Figure 6.11 
represents factor satisfaction of physicians compared to the 
other work groups. 
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Figure 6.12 represents factor satisfaction of 
ancillary personnel compared to the other work groups. 
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Figure 6.13 represents factor satisfaction of 
administrative personnel compared to the other work groups. 




52 




Figure 6.14 represents the group factor satisfaction 
indices of administrative personnel compared to ancillary 
personnel . 




a. Factor A (Local MID Staff and Services) 

This factor is the respondent's self-reported 
assessment of the attitude and responsiveness of the local MID 
staff and the services they provide, as well as, the quality 
of the relationship with the MID personnel ( Baroudi and 
Orlikowski ) • As seen in Figure 6.10, the administrative 
personnel displayed the highest level of satisfaction with a 
statistically significant difference at alpha = 0.05. The 
lowest level of satisfaction was displayed by the physicians, 
however, it is exceptionally high in comparison with their 
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responses to the other factors. The physicians were 
statistically significant at alpha = 0.05. 

b. Factor A ( Contractor , DMSSC) 

This factor represents the responsiveness and 
services of outside software programmers and management. It 
received the lowest level of satisfaction of any factor. All 
work groups rated this factor extremely low, with 
administrative personnel recording the highest level of 
satisfaction and the physicians scoring the lowest level of 
satisfaction. The administrative group showed a statistically 
significant higher difference while the physicians showed a 
statistically significant lower difference at alpha = 0.05. 

c. Factor B - Information Product 

This factor represents the users perception of the 
quality of output delivered by the information system (Baroudi 
and Orlikowski, 1988). The administrative group scored the 
highest level of satisfaction with a statistically significant 
difference at alpha = 0.05. The physicians recorded the 
lowest level of satisfaction with a statistically significant 
difference at alpha = 0.05. 

d. Factor C - Knowledge and involvement 

This factor represents the respondents' self- 
reported assessment of the quality of training provided, their 
understanding of the system, and their participation in its 
development (Baroudi and Orlikowski, 1988). There was a 
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statistically significant difference between the level of 
satisfaction of the administrative group and the physicians at 
alpha = 0.05. The administrative group and ancillary group 
rated the satisfaction level nearly the same. The physicians 
scored this factor the lowest of all the group factors. 

4. Time of System Use 

Igbaria and Nachman (1990) found that users with 
better computer background are likely to be more confident in 
their ability to use the information system and more satisfied 
with experience. Generally, users become more skilled with an 
information system over time. Therefore, the assumption is 
that the respondents who use the system the longest will be 
more experienced and record a greater level of satisfaction. 

The respondents were separated into subgroups of users 
with less than six months, six months to 11 months, and 
greater than 11 months experience. Almost sixty percent of 
the respondents had 11 months or greater experience with the 
AQCESS system. Figure 6.15 represents the breakdown of 
respondents by time of system use. 

The respondents who used the system more than or equal 
to 11 months generated the lowest level of user satisfaction. 
Conversely, the respondents with 6 to 11 months of system use 
recorded the highest level of satisfaction with the system. 
Regression analysis indicated no correlation (r < 0.02) 
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between the time of system use and level of user satisfaction 
with the system for any factor. 
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Figure 6.16 shows the level of satisfaction with AQCESS 
verses time of system use. 
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Breaking the work groups down by time of system use, 
there was still no correlation between time of system use and 
level of satisfaction of any factor by regression analysis. 
The following discussion will describe the level of user 
satisfaction for the work groups over time. 

The physicians with less than six months system use 
had a satisfaction index of 2.625. Unlike the ancillary and 
administrative groups, the physicians' level of satisfaction 
decreased to -3.25 between six to 11 months experience. The 
level of satisfaction for the physicians with more than 11 
months experience show a decrease at -6.36, a downward trend 
consistent with the other work groups. Figure 6.17 
illustrates the physician's level of satisfaction with respect 
to time of system use. 
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The ancillary group's level of satisfaction displays 
a similar trend to the administrative group. The satisfaction 
index for users with less than six months experience was 
slightly lower (18.17) than users with six to 11 months 
experience whose level of satisfaction rose to 19.8. 
Surprisingly, the level of satisfaction for users with more 
than 11 months of experience drops dramatically to 9.4. 
Figure 6.18 illustrates the ancillary group's level of 
satisfaction with respect to time of system use. 




Although the trends for both the administrative and 
the ancillary group's level of satisfaction are nearly the 
same, the administrative group's level of satisfaction amongst 
the user subgroups is more pronounced. The satisfaction index 
for users with less than six months experience was lower at 
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15.6 than users with six to 11 months experience whose level 
of satisfaction rises steeply to 22.35. The level of 
satisfaction for users with more than 11 months of experience 
drops sharply to 9.4. Figure 6.19 illustrates the 
administrative group's level of satisfaction with respect to 
time of system use. 
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VII. DESCRIPTIVE FINDINGS (SILAS B. HAYS HOSPITAL, FORT ORD) 



A. DEMOGRAPHIC FINDINGS 

Of the 57 respondents, 10 were male and 47 were female. 
The age of the respondents ranged from 19 to 65 with a mean of 
34 years old. 

1. Work Center Distribution 

The hospital department work centers reported were 
in/out-patient areas including; other (ie; Patient 
Administration Division, Quality Assurance, etc. ) and clinics 
(ie; OB/GYN, Pediatrics, etc.). Figure 7.1 reflects the 
distribution by work center. 
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Figure 7.1 
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2. Job Description Distribution 

The job descriptions reported are from a multitude of 
specific functions but can be categorized as: 

1. Technician - a hospital specialist functioning in 
a medical-technical capacity (laboratory 
technician, clinical technician, medic) 

2. Other - a person not listed above (hospital 
administration non-medical clinic staff) 

Figure 7.2 reflects the distribution by job description. 
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Figure 7 . 2 



3. Work Group Distribution 

For the purposes of this investigation, the study 
population was segregated according to the assumed primary use 
of the computer system. The categories provide a clear 
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separation of work groups from an organizational standpoint. 
These groups are used extensively throughout the study of the 
Silas B. Hays Hospital sample (Fort Ord). 

1. Administrative - use of the system for indirect 
medical purposes (patient registration, 
appointment scheduling and administrative 
reporting ) 

2. Ancillary - use of the system for direct medical 
support (prescription filling, laboratory specimen 
processing ) 

Figure 7.3 represents the population of users/respondents 
according to their main use of the system. 
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Figure 7.3 
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4. Education Distribution 

The level of education reported by the users 
represents a fairly well educated user population. 
Approximately 67 percent of the users have at least had some 
college education. 12 percent completed a Bachelors or 
Masters degree with 5 percent having worked on a Masters 
degree. Figure 7.4 represents the breakdown by education. 
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Figure 7.4 



B. COMPUTER SYSTEM USE 

The length of time users reported as having used the 
AQCESS computer system ranged from one month to 72 months with 
a mean of 19.4 months. Of the 57 respondents, 22 had used a 
computer system previously. 16 of the 57 users had used a 
health care information system before using AQCESS. Although 
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many of the respondents had experience with computers, this 
was the first exposure to a health care information system for 
72 percent of the users. 

C. SATISFACTION FINDINGS 

The survey results for user satisfaction are compared for 
the two work groups in three areas: 1) overall satisfaction, 
2) responses to individual questions, 3) three group factors: 
Information Management Division (IMD) staff and services; 
information system product; knowledge and involvement. The 
actual values of responses to individual questions reported in 
the survey may be significant in themselves. However, the 
relative scores are useful for comparison among various work 
groups, organizations and information systems. (Baroudi and 
Orlikowski, 1988) The administrative group composes nearly 97 
percent of the survey population and the remainder consists of 
the ancillary group. 

1. Overall Satisfaction (Fort Ord) 

Overall satisfaction is measured by the mean of the 
sum of the responses to the survey questions. The mean survey 
score index was 10.47 on a scale ranging from -39 to +39. 
Figure 7.5 presents a comparison of the overall satisfaction 
index of both groups and the overall mean satisfaction. The 
survey shows that the ancillary group displays the highest 
overall satisfaction index (13.0), with the administrative 
group recording the lower overall satisfaction (10.38). Using 
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a t-test, there was no significant difference between the 
ancillary group and the administrative group at alpha = 0.05. 

2 . Individual Questions 

The 13 questions on the survey require two responses 
each from the respondent. A number of the response scales are 
reverse scored to nullify the effects of a respondent who 
simply marks down one column of the questionnaire ( Baroudi and 
Orlikowski, 1988). Figure 7.6 shows the average level of 
satisfaction for individual questions by work group. 

Similar to the results discussed in chapter six, 
questions 1, 6 and 11 indicate a high level of user 
satisfaction. These questions deal with issues concerning the 
IMD staff. However, questions 2, 3 and 12 stand out as 

exhibiting a relatively low satisfaction index. Questions 2 
and 12 are concerned with DMSSC services and modifications and 
question 3, training. The following discussion will examine 
each question by work group. 

a. Question 1 - Relationship with the Information 
Management Division (IMD) Staff 
As noted earlier, the level of satisfaction towards 
the IMD staff is relatively high. Although not statistically 
significant at alpha = 0.05, the ancillary group showed the 
higher level of satisfaction between the two groups. 



65 




Individual Question Response 
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Figure 7.6 
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b. Question 2 - Processing of Requests for Changes 
to Existing Systems 

This question received the lowest satisfaction 
index rating. The response was neutral for the ancillary 
group and negative for the administrative group. A t-test 
indicated no statistically significant differences at alpha = 
0.05. 

c. Question 3 - Degree of Training Provided to Users 
The level of satisfaction for the ancillary group 

was higher than that of the administrative group but not 
statistically significant difference higher at alpha = 0.05. 

d- Question 4 - User's Understanding of the System 
There was no statistically significant difference 
between the administrative and ancillary groups at alpha = 
0.05. Again, the ancillary group displayed a higher 
satisfaction index. 

e. Question 5 - User's Feeling of Participation 

In response to this question, the administrative 
and the ancillary groups responded positively with no 
statistically significant difference at alpha = 0.05. 

f. Question 6 - Attitude of the Information 

Management Division (IMD) Staff 
The response to this question ( the second highest 
rated question) was favorable with no statistically 
significant difference at alpha = 0.05. 
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g. Question 7 - Reliability of Output Information 
The responses were similar with no statistically 

significant differences between the groups. 

h. Question 8 - Relevancy of Output Information ( to 
intended function) 

Both groups responded comparably without 
statistically significant differences. 

1. Question 9 - Accuracy of Output information 

The ancillary group was neutral regarding this 
item. The administrative group was slightly more positive, 
but there were no statistically significant differences at 
alpha = 0.05. 

j. Question 10 - Precision of Output Information 

The response by both groups was somewhat favorable 
with the administrative group's satisfaction index being a 
little higher. There were, however, no statistically 
significant differences. 

1c. Question 11 - Communication with the Information 
Management Division (IMD) Staff 

All questions concerning the IMD staff received 
high satisfaction levels from both groups (highest rated 
question response ) . Neither of the groups ' responses were 

statistically different by t-test at alpha = 0.05. 
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1. Question 12 - Time Required for New Systems 

Development 

The groups were nearly neutral regarding this 
issue. There were no statistically significant differences at 
alpha = 0.05. 

in. Question 13 - Completeness of Output 

Both groups responded similarly with no 
statistically significant differences at alpha = 0.05. 

3 . Grouped Factors 

As discussed in chapter VI, the questions are grouped 
into three factors that comprise user satisfaction. These 
factors: IMD staff and services, information system product, 
and knowledge and involvement, are calculated through averages 
of the component questions. IMD staff and services (Factor A) 
is sub-divided into two separate components: local IMD staff 
functions (IMD) and contractor/DMSSC functions (CON) . 

Neither of the work groups' satisfaction level for any 
of the factors were statistically different at significance 
level alpha = 0.05. Figure 7.7 represents the ancillary and 
administrative groups' level of satisfaction in relation to 
the group factors . 

a. Factor A (Local MID Staff and Services) 

In Figure 7.7, the ancillary group displayed the 
highest level of satisfaction but it was not statistically 
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significant different from the administrative group at alpha 



= 0.05. 
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Figure 7.7 



b. Factor A (Contractor Services) 

This factor received the highest level of 
satisfaction of any factor. The ancillary group was not 
statistically significant different from the administrative 
group at alpha = 0.05. 



c. Factor B - Information Product 



This factor is the second highest rated of the four 
and there is no statistically significant difference between 
the ancillary and administrative groups at alpha = 0.05. 
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d. Factor C - Knowledge and Involvement 

This factor showed no statistically significant 
difference between the level of satisfaction for the ancillary 
or administrative groups. 

4 . Time of System Use 

As stated in chapter VI, it is expected that the more 
experienced users will exhibit a greater level of 
satisfaction . 

Figure 7.8 displays the separation of users into 
subgroups having less than six months experience, six to 11 
months, and greater than 11 months. 
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Figure 7 . 8 

The ancillary group had no users with less than six 
months nor greater than 11 months of system use. Therefore 
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there is really no trend to report for this group. However, 
it is worthy to note that the two respondents displayed a high 
level of satisfaction with an index of 13.0. Figure 7.9 
depicts the ancillary users' satisfaction index versus the 
time of system use. 
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Figure 7.9 



The administrative group's level of satisfaction tends 
to increase with experience and time of use. Respondents, 
with less than six months use, recorded a satisfaction index 
of 5.3. As the users gained experience (six to 11 months), 
the level of satisfaction increased to 10.8. For users with 
greater than 11 months experience, the satisfaction index 
elevated to 12.06. Figure 7.10 represents the administrative 
users' satisfaction index versus the time of system use. 
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VIII 



COMPARISON OF RESULTS FROM FORT ORD AND PENSACOLA 



A. INTRODUCTION 

When the user satisfaction survey questionnaire is 
assessed within a single organization, the UIS scores are 
indicative of the general level of user satisfaction with a 
specific information system. The scores, positive or 
negative, are on its own an important finding. However, the 
scores are useful for comparisons across different users of 
the same information system (to pinpoint the problems 
particular users may be experiencing) , as well as for 
comparisons across various information systems (to highlight 
specific information systems that may be problematic) . 
(Baroudi and Orlikowski, 1988) . Silas B. Hays Army Hospital 
(Fort Ord) and Pensacola Naval Hospital are two different 
organizations with similar information systems (AQCESS) . This 
chapter will compare the user information satisfaction (UIS) 
measurements recorded from Fort Ord with the UIS measurements 
recorded from Pensacola. In a later chapter, these findings 
will be analyzed to pinpoint any agreements or disagreements 
amongst the work groups of these organizations. 
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B. OVERALL USER SATISFACTION 



The impressions of the respondents from Pensacola and 
Fort Ord towards their information system are comparable. 
Using a t-test, the satisfaction level of the two 
organizations, has no significant difference at alpha = 0.05. 
The Fort Ord respondents did register a slightly higher level 
of satisfaction at 10.47 which is inconsequential compared to 
Pensacola's overall satisfaction index at 9.02. Together their 
overall level of satisfaction is approximately 9.64. Figure 
8.1 describes the extent of the difference in the satisfaction 
level of the two organizations. 
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Figure 8 . 1 



The combined work groups' overall satisfaction was 
determined by combining the mean of the sum of their 
corresponding satisfaction indices measured from the survey 
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questions of the 57 respondents from Fort Ord and the 174 
respondents from Pensacola. The survey results, for the 
combined administrative group, display the highest overall 
satisfaction index at 11.87, followed by the combined 
ancillary group's level of satisfaction at 9.46. Fort Ord did 
not have any survey responses from physicians. Thus the 
overall satisfaction level, -4.16, for the physicians' is 
obtained solely from the Pensacola physicians. As noted in 
chapter six, the physician's level of satisfaction has a 
statistically significant lower difference than the other 
groups at alpha = 0.05. Figure 8.2 describes the overall 
level of satisfaction of the work groups from Fort Ord and 
Pensacola . 
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1 . 



Fort Ord ' s and Pensacola's Individual Question 



Responses 

The method used to calculate the values of the 
individual question responses was described in earlier 
chapters. Together, Fort Ord and Pensacola display a similar 
pattern of satisfaction levels for individual question 
responses as they did separately. Still, questions 1, 6 and 
11 (deals with issues concerning MID/IMD) show the highest 
level of satisfaction. Consistent with previous reports. 
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The combined work groups display a similar pattern of 
satisfaction levels for individual question responses as 
previously noted. Questions 1, 6 and 11 consistently rate as 
having the highest level of satisfaction, while questions 2, 
3 and 12 consistently rate as having the lowest level of 
satisfaction by the respondents of the combined work groups. 
Each question will be discussed further in the following 
sections. Since the physician's responses are solely from 
Pensacola and are not changed by combining the work group 
responses from Fort Ord, they will not be discussed as part of 
the work group. They will be reflected in the overall 
comparison of the individual question responses between Fort 
Ord and Pensacola. Figure 8.4 illustrates the satisfaction 
index for individual questions by combined work groups. 
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Figure 8.4 
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a. Question 1 - Relationship with the Information 

Management (MID/IMD) Staff 
As noted earlier, the level of satisfaction towards 
the MID/IMD staff is relatively high. Pensacola's overall 
level of satisfaction was higher than Fort Ord's. Their 
combined administrative group and combined ancillary group 
responses to this question are practically the same. Overall, 
Pensacola shows a slightly higher level of satisfaction for 
their MID staff. There was no statistically significant 
difference at alpha = 0.05 for the combined work groups or 
hospitals . 

b. Question 2 - Processing of Requests for Changes 
to Existing Systems 

The responses to this question received the lowest 
satisfaction index rating. Both combined work groups and 
hospitals responses were negative towards the processing of 
requests for outside services. However, Fort Ord displayed 
a statistically significant lower difference lower at alpha = 
0.05. 

c. Question 3 - Degree of Training Provided to Users 
The level of satisfaction for this item was rated 

next to the lowest by both the combined work groups and 
hospitals. There was no statistically significant difference 
at alpha = 0.05. 
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d- Question 4 - User's Understanding of the System 
Fort Ord and the combined administrative group 
displayed the highest level of satisfaction toward 
comprehension of their system. There was no statistically 
significant difference between at alpha = 0.05. 

e. Question 5 - User's Feeling of Participation 

The work groups and hospitals show relatively the 
same level of satisfaction for involvement and participation 
in services and the functioning of the system. There was no 
statistically significant difference at alpha = 0.05. 

f. Question 6 - Attitude of the Information 

Management ( MID/IMD ) staff 
Fort Ord the combined administrative group tend to 
show a slightly higher level of satisfaction toward their 
MID/IMD staffs. There was no statistically significant 
difference at alpha = 0.05. 

g. Question 7 - Reliability of Output Information 
Fort Ord perceives slightly higher degree of 

satisfaction for reliability of output information. The 
combined work groups ' level of satisfaction seems were 
comparable. There was no statistically significant difference 
at alpha = 0.05. 
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h. Question 8 - Relevancy of Output Information ( to 
intended function) 

Fort Ord and the combined administrative group had 
a higher level of satisfaction regarding the relevancy of 
output information. There was no statistically significant 
difference at alpha = 0.05. 

i. Question 9 - Accuracy of Output Information 

Fort Ord and the combined administrative group tend 
to be more confident in the accuracy of the output 
information. There was no statistically significant 

difference at alpha = 0.05. 

j. Question 10 - Precision of Output Information 
The precision of the output information is more 

favorable to Fort Ord and the combined administrative group. 
There was no statistically significant difference at alpha = 
0.05. 

Jc. Question 11 - Communication with the Information 
Management Division (MID/IMD) Staff 

The manner and methods of information exchange 
between the users and MID/IMD staff are highly valued by the 
hospitals and the combined work groups. There was no 
statistically significant difference by t-test at alpha = 
0.05. 
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l. Question 12 - Time Required for New Systems 

Development 

According to the hospitals' and combined work 
groups' level of satisfaction, the elapsed time for new 
systems development by DMSSC or contractors is practically 
unacceptable. Fort Ord may be slightly more tolerant than 
Pensacola for development of new systems. There was no 
statistically significant differences at alpha = 0.05. 

m. Question 13 - Completeness of Output 

Almost an even level of satisfaction amongst the 
hospitals and the combined work groups. They show some 
satisfaction towards the comprehensiveness of the output 
information. There was no statistically significant difference 
at alpha = 0.05. 

2 . Combined Grouped Factors 

Fort Ord's and Pensacola's overall responses for the 
grouped factors are joined together to form overall combined 
grouped factors. These overall combined grouped factors are 
broken down to form overall work group factors. Refer to 
chapters six and seven for further discussion on group 
factors. The following sections will discuss the overall 
combined factors and combined work group factors for Fort Ord 
and Pensacola. The physicians will not be included in the 
discussion compared by work group. Neither the hospitals' or 
work groups' satisfaction level for any of the factors were 
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statistically different at significance level alpha = 0.05. 
Figure 8.5 represents the overall grouped factors for Fort Ord 
and Pensacola. 
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Figure 8.6 represents the overall factors by work 



groups . 



GROUP FACTORS 
FT ORD AND PENSACOLA COMBINED 

































« : : 




Wk 




















% 




i 










0^ 






j 













Foctof A FoctcK A B C 

KC Cortrocfor 



PHYSOAN ^ AOMNSTRAIIVE M ANCUARY 



Figure 8 . 6 
83 





a. Factor A (Local MID/IMD Staff and Services) 

Fort Ord and the combined ancillary group has the 
highest level of satisfaction for the MID/IMD staff and 
services. This factor was rated the highest overall. There 
was no statistically significance difference at alpha = 0.05. 

b. Factor A (Contractor Services) 

Consistent with other results, this factor received 
the lowest level of satisfaction of any factor. There was no 
statistically significant difference at alpha = 0.05. 

c. Factor B - Information Product 

This factor is the second highest rated of the four 
factors. There was no statistically significant difference at 
alpha = 0.05. 

d. Factor C - Knowledge and involvement 

This factor showed no statistically significant 
difference between the level of satisfaction among the work 
groups. The administrative work group and Fort Ord rated 
knowledge and involvement slightly better than the other 
groups . 
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IX. 



COMPARISON OF RESULTS FROM SILAS B. HAYS HOSPITAL, 



NAVAL HOSPITAL, PENSACOLA (AQCESS) AND NAVAL H3SPTIAL, 
CHARLESTON (CHCS) 

A. INTRODUCTION 

This chapter compares AQCESS and the CHCS user 
satisfaction findings. Both systems are fully described in 
chapters II and III. The survey results for user satisfaction 
are compared with respect to each site, system versus system 
and each work group. The criteria are as follows: 1) overall 
satisfaction, 2) responses to individual questions, 3) three 
group factors: MID staff and services; information system 

product; knowledge or involvement. ( Baroudi and Orlikowski, 
1988) 

1. Overall Satisfaction (Pensacola, Fort Ord, Charleston) 
The mean overall satisfaction index of the combined 
sites is 11.07 on a scale ranging from -39 to +39. Charleston 
(CHCS site) displays the highest overall satisfaction at 
11.25, followed by Fort Ord at 10.47. Pensacola registers the 
least overall satisfaction at 9.025. Using a t-test, the 
level of satisfaction was not statistically significant at 
alpha = 0.05. Figure 9.1 illustrates the overall satisfaction 
index for each site. 
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OUERALL SATISFACTION 

PENSACOLA vs n ORO v> CHARLESTON 




Figure 9 . 1 



2 . Individual Questions 

Figure 9.2 displays the average level of satisfaction 
for each site. Responses to questions 1, 6 and 11 indicate a 
high level of user satisfaction. As stated earlier, these 
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questions are concerned with the Management Information 
(MID/IMD) staff. Consistent with previous findings, responses 
to questions 2, 3 and 12 stand out as exhibiting a relatively 
low satisfaction index. Questions 2 and 12 are concerned with 
outside contractors and/or DMSSC services and question 3, 
training. The following discussion will further elaborate on 
each question. 

a. Question 1 - Relationship with the Management 
Information (MID/IMD) Staff 

The level of satisfaction towards the MID staff is 
relatively high (second highest satisfaction index). 
Charleston shows the highest level of satisfaction of the 
three sites, closely followed by Pensacola. Fort Ord reports 
the lowest satisfaction index. There is no statistically 
significant difference at alpha = 0.05. 

b. Question 2 - Processing of Requests for Changes 
to Existing Systems 

All three sites reports a negative level of 
satisfaction for time to process requests for changes to 
existing system. This, the lowest scoring response, is 
consistent with previous descriptions. Pensacola gave it the 
lowest rating, followed by Charleston and Fort Ord, 
respectively. The level of satisfaction for Pensacola was 
statistically significant difference lower versus Charleston's 
level of satisfaction at alpha = 0.05. 
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c- Question 3 - Degree of Training Provided to Users 
The level of satisfaction of each site was positive 
but nearly neutral at Pensacola and Fort Ord. Charleston 
reported the highest satisfaction index. However, Pensacola 
showed a statistically significant lower difference to 
Charleston's level of satisfaction at alpha = 0.05. 

d. Question 4 - User’s Understanding of the System 
Fort Ord and Charleston were nearly equal in 

satisfaction, while Pensacola reported a lower satisfaction 
index. There is no statistically significant difference at 
alpha =0.05. 

e. Question 5 - User's Peeling of Participation 
Charleston rated this item a little higher than 

Fort Ord or Pensacola, which reported a similar level of 
satisfaction. However, Pensacola showed a statistically 
significant lower difference than Charleston's level of 
satisfaction at alpha = 0.05. 

f. Question 6 - Attitude of the Management 
Information (MID/IMD) Staff 

This was the highest rated satisfaction index of 
all 13 indices. Charleston's response rated very high while 
Pensacola followed closely. Fort Ord's response was rated 
lower, but positive. Pensacola's and Fort Ord's level of 
satisfaction show a statistically significant lower difference 
to Charleston's level of satisfaction at alpha = 0.05. 
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g. Question 7 - Reliability of Output Information 
Fort Ord rated this item higher than either 

Pensacola or Charleston, which responded similarly. There is 
no statistically significant difference at alpha = 0.05. 

h. Question 8 - Relevancy of Output Information ( to 
intended function) 

Again, Fort Ord rated this item higher than either 
Pensacola or Charleston, which responded similarly. There is 
no statistically significant difference at alpha = 0.05. 

X. Question 9 - Accuracy of Output Information 

The response to this question resulted in a 
relatively high satisfaction index for Charleston which is 
followed by Fort Ord. Pensacola's level of satisfaction shows 
a statistically significant lower difference to Charleston's 
level of satisfaction at alpha = 0.05. 

j. Question 10 - Precision of Output Information 

Pensacola and Charleston reported similar levels of 
satisfaction. Fort Ord, although not to any large degree, 
reported a higher level of satisfaction. There is no 
statistically significant difference at alpha = 0.05. 

Tc. Question 11 - Communication with the Management 
Information (MID/IMD) Staff 

This item is rated among the top three in levels of 
satisfaction. Charleston has the highest index. Fort Ord is 
second and Pensacola, although positive, reports the lowest 
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level of satisfaction. There is no statistically significant 
difference at alpha = 0.05. 

l. Question 12 - Time Required for New Systems 

Development 

This item is rated as the second lowest among the 
13 responses. Charleston's response results in a negative 
index which has statistically significant lower difference at 
alpha = 0.05. Fort Ord has the highest level of satisfaction. 

m. Question 13 - Completeness of Output 

The responses to this question are relatively 
close. Charleston demonstrates the lowest level of 

satisfaction while Fort Ord has the highest. There is no 
statistically significant difference at alpha = 0.05. 

3 . Grouped Factors 

The following sections will discuss the overall 
combined factors by sites. Fort Ord's and Pensacola's levels 
of satisfaction displays a statistically significant lower 
difference to Charleston's level of satisfaction at alpha = 
0.05. Figure 9.3 represents each site's level of satisfaction 
with respect to the grouped factors. 

a. Factor A (Local MID/IMD Staff and Services) 

As seen in Figure 9.3, Charleston displays the 
highest level of satisfaction, followed by Fort Ord. The 
lowest level of satisfaction is reported by Pensacola. Both 
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GROUP FACTORS 
PENSACOLA vs FT ORD vs CHARLESTON 
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Figure 9 . 3 

Fort Ord ' s and Pensacola's levels of satisfaction results in 
a statistically significant lower difference to Charleston's 
at alpha = 0.05. 

b. Factor A ( Contractor , DMSSC) 

This factor received the lowest level of 
satisfaction of any group factor. All sites rated this factor 
extremely low, with Pensacola and Fort Ord almost rating it 
nearly neutral. There is no statistically significant 
difference at alpha = 0.05. 

c. Factor B - Information System Product 

This was the second highest rated factor. All 
three sites report similarly, with Fort Ord recording the 
highest level of satisfaction. There is no statistically 
significant difference at alpha = 0.05. 
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d- Factor C - Knowledge and Involvement 



The second lowest rated factor, all sites report 
similarly. Charleston's level of satisfaction is the highest, 
followed by Ford Ord's. Pensacola's level of satisfaction is 
the lowest. There is no statistically significant difference 
at alpha = 0.05. 

4. Overall Satisfaction (CHCS vs AQCESS) 

The average overall satisfaction index of CHCS and 
AQCESS is 10.54. CHCS reports the highest overall 
satisfaction (11.25), with AQCESS at 10.13. Using a t-test, 
the difference in level of satisfaction is not statistically 
significant at alpha = 0.05. Figure 9.4 illustrates the 
overall satisfaction index for each system. 

5 . Individual Questions 

Figure 9.5 displays the average level of satisfaction 
for each site. Again, responses to questions 1, 6 and 11 
indicate a high level of user satisfaction. (MID/IMD) Also 
consistent with previous findings, responses to questions 2, 
3 and 12 stand out as exhibiting a relatively low satisfaction 
index. Questions 2 and 12 are concerned with outside 
contractors and/or DMSSC services and question 3, training. 
The following discussion will further examine each question. 
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a. Question 1 - Relationship with the Management 
Information (MID/IMD) Staff 

The level of satisfaction towards the MID staff is 
relatively high (second highest satisfaction index). CHCS 
users have the higher level of satisfaction, however, there is 
no statistically significant difference at alpha = 0.05 

between the two indices. 

b. Question 2 - Processing of Requests for Changes 
to Existing Systems 

This, the lowest scoring response, received a 
negative index from both system users. CHCS users reported 
the lowest satisfaction rating, however, there was no 
statistically significant difference at alpha = 0.05. 

c. Question 3 - Degree of Training Provided to Users 
The level of satisfaction for each system was 

positive but nearly neutral for AQCESS users. CHCS showed a 
statistically significant lower difference at alpha = 0.05. 

d. Question 4 - User's Understanding of the System 
CHCS was rated higher in satisfaction than AQCESS 

for this item. There was, however, no statistically 
significant difference between the indices at alpha = 0.05. 

e. Question 5 - User's Feeling of Participation 
CHCS users reported a higher index than AQCESS 

users. AQCESS showed a statistically significant lower 
difference at alpha = 0.05. 
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f. Question 6 - Attitude of the Management 
Information (MID/IMD) Staff 

Again, this was the highest rated satisfaction 
index of all 13 indices. CHCS users' response was rated very 
high while AQCESS users, who yielded a positive rating, were 
statistically significant difference lower at alpha = 0.05. 

g. Question 7 - Reliability of Output Information 
The satisfaction indices for this item were 

positive and virtually the same. 

h. Question 8 - Relevancy of Output Information ( to 
intended function) 

Both CHCS and AQCESS users gave positive responses. 
There was no statistically significant difference at alpha = 
0.05. 

1. Question 9 - Accuracy of Output Information 

The response to this question resulted in a 
relatively high satisfaction index for CHCS users. However, 
AQCESS users show a statistically significant difference lower 
at alpha = 0.05. 

j. Question 10 - Precision of Output Information 

The CHCS and AQCESS users responded similarly with 
no statistically significant difference at alpha = 0.05. 
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Jc. Question 11 - Communication with the Management 
Information ( MID/IMD ) Staff 

Rated third highest index overall, there was no 
statistically significant difference at alpha = 0.05. 

l. Question 12 - Time Required for New Systems 

Development 

This item is rated as the second lowest among the 
13 indices. CHCS users' response resulted in a negative 
index, while AQCESS users showed a statistically significant 
higher difference at alpha = 0.05. 

m. Question 13 - Completeness of Output 

The responses to this question were positive with 
no statistically significant difference at alpha = 0.05. 

6 . Grouped Factors 

The following sections will discuss the overall 
combined factors for the respondents of the CHCS and AQCESS 
systems. Figure 9.6 represents each site's level of 
satisfaction in relation to group factors. 

a. Factor A (Local MID/IMD Staff and Services) 

As seen in Figure 9.6, CHCS respondents report a 
much higher level of satisfaction for their local staff and 
services. AQCESS respondents' level of satisfaction show a 
statistically significant lower difference at alpha = 0.05. 
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GROUP FACTORS 
CHCS vs ACCESS 




VM CHCS ^ ACX^ISS 



Figure 9.6 

h. Factor A (Contractor, DMSSC) 

This factor received the lowest level of 
satisfaction of any group factor. Both systems' users rate 
this factor low, with CHCS being statistically significant 
difference lower at alpha = 0.05. 

c- Factor B - Information System Product 

CHCS and AQCESS users responded essentially the 
same. There is no statistically significant difference in 
levels of satisfaction from either systems' respondents at 
alpha = 0.05. 

d. Factor C - Knowledge and Involvement 

The users of the CHCS system display a slightly 
higher level of satisfaction than the AQCESS users. There is 
no statistically significant difference at alpha = 0.05. 
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X. 



COMBINED DESCRIPTIVE FINDINGS OF CHCS AND AQCESS 



A. INTRODUCTION 

The AQCESS and CHCS systems, although similar in many 
areas, are functionally different systems. However, the 
survey responses from both systems are joined to assist in the 
analysis of findings across these systems. Chapter IX 
compared the Charleston Naval Hospital ' s CHCS system to 
Pensacola Naval Hospital's and Fort Ord Army Hospital's AQCESS 
systems. This chapter will examine the combined results of 
the survey responses from the hospitals. Figure 10.1 shows a 
breakdown of the respondents by work groups from the combined 
hospitals. 



AQCESS AND CHCS 

bY WORK GROUPS 
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Figure 10.1 
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B. OVERALL USER SATISFACTION FOR THE COMBINED INFORMATION 



SYSTEMS 

The overall satisfaction is measured by the mean of the 
sum of the responses to the survey questions from the 
hospitals. A t-test the overall satisfaction level of the 
ancillary group and administrative group has no statistically 
significant difference at alpha = 0.05. The physicians are 
statistically significant difference lower response at alpha 
= 0.05. The survey shows that the ancillary group displays 
the highest overall satisfaction index at 11.47. When 
compared to the overall satisfaction index of the 
administrative groups level at 11.43, the ancillary group and 
administrative group overall satisfaction for their 
information systems are approximately equal. The physicians 
level of satisfaction index for the systems are the lowest at 
1.35. The overall satisfaction level was 10.14. Figure 10.2 
illustrates the combined work groups level of satisfaction for 
the AQCESS and CHCS hospital information systems. 

1 . Individual Question Responses for AQCESS and CHCS 

The method used to calculate the values of the 
individual question responses is described in earlier 
chapters. Together, the CHCS and AQCESS systems show the same 
trend as the other comparisons. Questions 1, 6 and 11 (deals 
with issues concerning MID/IMD) show the highest level of 
satisfaction. Consistent, questions 2, 3 and 12 (deals with 
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DMSSC services and management) have the lowest rated indices. 
Figure 10.3 displays the overall level of satisfaction for 
individual questions for the CHCS and AQCESS combined work 
groups. 
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In the comparisons of one work group versus the remaining 
work groups, the patterns of satisfaction levels for 
individual question responses are similar. Questions 1, 6 and 
11 consistently rate the highest level of satisfaction, while 
questions 2, 3 and 12 consistently rate the lowest level of 
satisfaction by the respondents of the combined work groups. 
Each question will be discussed further in the following 
sections. Figure 10.4 displays the responses to individual by 
physicians compared to the remainder of respondents; Figure 
10.5 ancillary compared to the other groups; Figure 10.6 
administrative compared to the other groups. 



Individual Question Response 
Physicians vs Others 




PhysWon ^ Othtra 



Figure 10.4 
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a. Question 1 - Relationship with the Information 
Management ( MID/IMD ) Staff 
As noted earlier, the level of satisfaction towards 
the MID/IMD staff is relatively high. The level of 
satisfaction is higher for the ancillary group than the others 
but not much higher than the administrative group. The 
physicians show a statistically significant lower difference 
at alpha = 0.05. 

jb. Question 2 - Processing of Requests for Changes 
to Existing Systems 

The responses to this question received the lowest 
satisfaction index rating. All work groups' responses were 
negative towards the processing of requests for outside 
services to make changes to the existing systems. However, 
the level of satisfaction of the administrative work group 
showed a statistically significant higher difference at alpha 
= 0.05. The ancillary group was much lower followed by the 
physicians . 

c. Question 3 - Degree of Training Provided to Users 
The ancillary group shows a higher level of 
satisfaction for the amount of training they have received on 
using. The physicians expressed the lowest level of 
satisfaction for the degree of training they have received. 
The level of satisfaction for the ancillary group shows a 
statistically significant higher difference and the physicians' 
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level of satisfaction has a statistically significant lower 
difference lower at alpha = 0.05. 

d. Question 4 - User's Understanding of the System 
The ancillary group displayed the highest level of 

satisfaction toward comprehension of their systems. The level 
of satisfaction for the ancillary group has a statistically 
significant higher difference at alpha = 0.05. The level of 
satisfaction for the physicians showed a statistically 
significant lower difference at alpha = 0.05. 

e. Question 5 - User's Peeling of Participation 

The ancillary group showed the highest level of 
satisfaction for involvement and participation in services and 
the functioning of the system followed closely by the 
administrative group. The ancillary group's level of 

satisfaction showed a statistically significant higher 
difference while the physicians' level of satisfaction showed 
a statistically significant lower difference at alpha = 0.05. 

f. Question 6 - Attitude of the Information 

Management (MID/IMD) Staff 
Even though the physicians showed level of 
satisfaction that was a statistically significant lower 
difference at alpha = 0.05, all work groups responded 

favorably to the attitudes of the MID/IMD staffs. The 
ancillary groups' level of satisfaction was the highest, 
followed closely by the administrative group. 
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g. Question 7 - Reliability of Output Information 
The perceptions of the work groups for reliability 

of output information were relatively low. The administrative 
group's level of satisfaction was slightly higher than the 
ancillary work group's. The physicians' level of satisfaction 
was extremely low and showed a statistically significant lower 
difference at alpha = 0.05. 

h. Question 8 - Relevancy of Output Information ( to 
intended function) 

Regarding the relevancy of output information, the 
administrative group had a higher level of satisfaction, 
followed closely by the ancillary group. The physicians' 
level of satisfaction showed a statistically significant lower 
difference at alpha = 0.05. 

i. Question 9 - Accuracy of Output Information 

All work groups showed positive degree of 
confidence in the accuracy of the output information. The 
level of satisfaction for the ancillary group is slightly 
higher but nearly even with the satisfaction level of the 
administrative group. The physicians level of satisfaction is 
the lowest. There was no statistically significant difference 
in the level of satisfaction for the groups. 

j. Question 10 - Precision of Output Information 
All work groups' level of satisfaction for the 

precision of the output information was relatively low. The 
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administrative group's level of satisfaction was the highest, 
followed closely by the ancillary group. The physicians' 
level of satisfaction is the lowest. There was no 
statistically significant difference in level of satisfaction 
at alpha = 0.05. 

k. Question 11 - Communication with the Information 

Management Division (MIS/IMD) 

The manner and methods of information exchange 
between the users and the MID/IMD staffs was highly valued by 
the work groups. The ancillary group's level of satisfaction 
was the highest. Slightly closer were the administrative 
group and physicians. This was the highest level of 
satisfaction recorded by the physicians. There was no 
statistically significant difference at alpha = 0.05. 

l. Question 12 - Time Required for New Systems 

The work groups' level of satisfaction, for time 
required to develop new systems by DMSSC or contractors, was 
very low. The administrative group's level of satisfaction 
for the systems was the highest, followed by the ancillary 
group. The physicians' level of satisfaction was extremely 
low. The administrative group's level of satisfaction 
revealed a statistically significant higher difference and the 
physicians' level of satisfaction revealed a statistically 
significant lower difference at alpha = 0.05. 
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m. Question 13 - Completeness of Output 



The work groups showed some satisfaction towards 
the comprehensiveness of the output information. The 
administrative group's level of satisfaction was the highest, 
followed by the ancillary group. The physicians recorded the 
lowest level of satisfaction. The administrative group's 
level of satisfaction showed a statistically significant 
higher difference at alpha = 0.05. 

2 . Combined Grouped Factors 

The following sections will discuss the overall 
combined factors by work groups. The physicians' level of 
satisfaction displayed a statistically significant lower 
different for the four factors at alpha = 0.05. Figure 10.7 
represents the work groups' level of satisfaction with respect 
to the sub-total factors. Figure 10.8 displays the 
satisfaction level for the factors by the physicians compared 
to the remainder of the groups; Figure 10.9 ancillary compared 
to the others; Figure 10.10 administrative compared to the 
others . 

a. Factor A (Local MID/IMD Staff and Services) 

The ancillary group had the highest level of 
satisfaction for the MID/IMD staff, followed closely by the 
administrative group. Although this factor was rated as being 
the highest level of satisfaction by the work groups, the 
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physicians' level of satisfaction showed a statistically 
significant lower difference at alpha = 0.05. 
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b. Factor A (Contractor Services) 

The type and quality of services rendered by DMSSC 
or contractors have been consistently rated low by the work 
groups in all comparisons. The ancillary group showed the 
only positive level of satisfaction amongst the groups, but 
was nearly neutral. The administrative group followed with a 
negative rating for level of satisfaction and the physician 
with the lowest. The physicians' level of satisfaction showed 
a statistically significant lower difference at alpha = 0.05. 
c- Factor B - Information Product 

This factor was rated positive by all groups. The 
highest level of satisfaction was given by the ancillary group 
which was comparable to the administrative group's. The 
physicians' level of satisfaction for the information product 
was the lowest and showed a statistically significant lower 
difference at alpha = 0.05. 

d. Factor C - Knowledge and Involvement 

The administrative group's level of satisfaction 
for knowledge and involvement was the highest, closely 
followed by the ancillary group. The physicians showed a 
statistically significant lower difference, at alpha = 0.05. 
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XI. ANALYSIS OF FINDINGS 



A. INTRODUCTION 

Performance evaluation is a crucial part of the management 
of an information system. The amounts of resources invested 
and the organization's dependence on information processing 
warrant the most efficient, effective use of the information 
system. However, without direction or objective, this 
evaluation is simply an assessment of the current operation of 
the information system. (Mensching and Adams, 1991) The 
purpose of this research is to measure and document the user 
information satisfaction of the Automated Quality of Care 
Evaluation Support System (AQCESS) at the Naval Hospital, 
Pensacola, Florida and Silas B. Hays Hospital, Fort Ord, 
California. These findings from the AQCESS system will be 
compared to those from the Composite Health Care System ( CHCS ) 
at the Naval Hospital, Charleston, South Carolina (Hurd, 
1991). The results should not be viewed as being a conclusive 
evaluation of the AQCESS or CHCS systems, but provide a 
benchmark for analyzing user satisfaction and identifying 
possible areas of discontent. To obtain a deeper 
understanding of the identified and highlighted symptoms of 
discontent, interviews should be conducted to examine the 
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development and operating procedures of the particular work 
group involved in order to determine the underlying causes by 
adding context and history. Upon completion of this process, 
it may become possible to make refinements and tuning to the 
system to increase its effectiveness and, perhaps secondarily, 
its efficiency. 

The first section will analyze the survey findings in 
chapters VI. through X. It will primarily focus on the 
differences and similarities among: work groups 
(administrative, ancillary, and physicians) at Pensacola; work 
groups at Fort Ord; combined work groups of Pensacola and Fort 
Ord; CHCS and AQCESS systems at Charleston, Pensacola and Fort 
Ord; combined work groups of Pensacola, Fort Ord and 
Charleston. 

B. PENSACOLA OVERALL SATISFACTION 

No one can deny that the AQCESS systems have not been a 
benefit to those sites that have this system. Automation has 
improved their overall efficiency and effectiveness. (NMDSC, 
1991). Pensacola Naval Hospital has benefitted tremendously 
from the AQCESS system. In order for the AQCESS system to 
continue to be a successful computer system, it must provide 
the needed information required by the users ( Olson and 
Baroudi, 1983). The following discussion will attempt to 
analyze the levels of dissatisfaction that are statistically 
significant by the respondents from Pensacola. Although not 
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conclusive, it should be used as a baseline measure to help 
verify areas in the AQCESS system that may require 
improvements or fine tuning. 

1 . Combined Groups 

The overall satisfaction index is a general 
description of the user satisfaction with the system. The 
survey average satisfaction index is 9.64. The respondents 
average overall satisfaction is characterized as only being 
marginally satisfied. 

2. Comparison of Work Groups 

The physicians' level of satisfaction showed a 
statistically significant lower difference than the 
administrative and ancillary work groups. However, the 
administrative group was somewhat more satisfied than the 
ancillary group, but only slightly. 

The physicians' group was dissatisfied with the 
system, which is indicated by their low average satisfaction 
index at -4.16. Generally, the only way the physicians 
legitimately interface with the system is by reviewing 
statistics of their performance, which are compiled by the 
Quality Assurance personnel. To a lesser degree, the 
physicians interact with the system through the Ad Hoc Report 
Generator module in the system. The physicians mainly view 
the system as a means of reporting their performance of 
patient care, which, according to their perspective, may not 
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be accurate. Therefore, any inaccuracies in a physician's 
performance report is viewed negatively by that physician and 
potentially leads to conflict with those who generate the 
reports. So, it is reasonable to assume that their overall 
low level of satisfaction can be attributed to their limited 
interface with the AQCESS system. 

Ancillary made up 39 percent of the personnel surveyed. 

Their average satisfaction index is 11.49, slightly lower 
than the administrative group. Most of the users in this work 
group are technicians who use the system for direct medical 
care (e.g., canceling and editing disposition data, patient 
history, diagnosis and procedure data record tracking and 
reporting) . As the large number of items processed each day 
in these areas increase, the response time for processing 
decreases to unacceptable levels. It has become routine for 
ancillary personnel to wait more than five minutes to retrieve 
records. When time is critical for patient care, this waiting 
is a source of frustration for ancillary personnel. 

The administrative group recorded the highest overall 
satisfaction score. Administrative personnel use the system 
for purposes other than direct conduct of patient care. With 
a few exceptions, the work performed on the system can be 
accomplished without the pressure of a limited and specific 
time constraints. Therefore, they can use the computer system 
at a comparatively slower pace than the other work groups. 
Consequently, interviews with most administrative personnel 
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revealed concern about the slow computer response time. 
Additionally, interviews revealed a lack of trust for the 
integrity of the output information by the system. Some 
administrative personnel argued that their inability to 
correct data, once it had been entered into the system, made 
the information suspect. 

3. Significant Individual Question Differences and 

This section will discuss only those responses to the 
questions with significant differences among the work groups. 
The physicians' level of satisfaction for all questions showed 
statistically significant lower differences than those of the 
other work groups. A low level of satisfaction for all 
questions by the physicians can be expected due to their lack 
of involvement and understanding of the system coupled with 
their limited role as a user of the system. Those questions 
that are statistically significant for the ancillary and 
administrative personnel will be discussed in the following 
sections. Since the physicians' responses to the questions 
are statistically significant in every case, only those 
questions, which will be helpful in gaining a better 
appreciation of the degree satisfaction, will be analyzed. 
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a. Relationship with the Management Information 



Department fMID> 

The results of the survey showed the physicians to 
be the least satisfied with the services of by the MID staff. 
Ironically, their level of satisfaction for this question was 
the highest of all their responses. Typically, unless it is 
to report inaccuracies in their performance report, which is 
generated by the system, the physicians' seldom communicate 
with the MID staff. If this was the only method in which the 
physicians and the MID staff communicated, then the 
physicians' relatively low level of satisfaction with the 
staff can be expected. 

b. Processing of Requests for Changes to the Existing 

System 

DMSSC is responsible for providing the support and 
services for the AQCESS system. Request for changes must go 
to DMSSC for approval and funding. If enough AQCESS sites 
request or support the change, then an upgrade to the system 
will be made. This upgrade, after testing, is then released 
as an improved version of the previous AQCESS system. The 
latest version for testing is AQCESS 7.0, which will take a 
year to release, if successful. (ADPAQCESS NMDSC, 1991) 
Because AQCESS is under centralized control (DMSSC) , changes 
to AQCESS will not occur rapidly. 
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c. User's Understanding of the System 

Users with better computer background and training 
are likely to be more confident in their ability to use the 
system and are more satisfied with the experience. The work 
groups that reported a low level of satisfaction in user 
training (physicians in particular) demonstrated a 
significantly lower level of satisfaction with their 
understanding of the system. 

d. User's Peeling of Involvement 

The presumption is that user involvement will lead 
to better understanding of the system and develop a system 
tailored to meet specific needs. Therefore, users will be 
more satisfied with the system than if they had not been 
involved. (Baroudi, Olson and Ives, 1986) Physicians are 
rarely involved with the system. So, their low of level 
satisfaction is expected. Consequently, administrative 
personnel are involved daily in the operations of the system 
and they feel involved with the system. 

e- Relevancy of Output Information 

When the output information is less than what the 
user perceives, the user will be less satisfied with the 
system (Conrath and Mignen, 1990). All groups have a low 
perception of the output information. 
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f. Accuracy of Output Information 

Physicians often question the accuracy of their 
statistical performance reports. If the information is 
inaccurate, then the system may be more likely to be blamed 
for the mistake than the person who inputs the information. 
This may be the main source of discontent for the physicians. 

g. Completeness of Output 

The comprehensiveness of the output information 
relates to the relevancy and accuracy of the information. If 
the user perceives the information to be lacking, the user 
will more likely think the information is inaccurate and/or 
irrelevant. It is not surprising to find the administrative 
group scoring significantly higher than the other groups on 
this question. The administrative group does not use this 
information directly. 

4. Grouped factors 

The overall management of the AQCESS system is 
centrally controlled by DMSSC who are responsible for the 
software support for the system. Local support services are 
provided by the local MID. Therefore, the single factor, MID 
staff and services, found in the original study by Ives, 
Olson, and Baroudi (1988), was altered to make two separate 
factors: local MID staff and services and Contractor's 

services . 
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The physicians scored the four factors statistically 
significant lower and the administrative group scored the four 
factors statistically significant higher for level of 
satisfaction. As alluded to earlier, due to the physicians' 
limited role and involvement with the system, their low level 
of satisfaction is predictable. The administrative group is 
not as constrained for time as the ancillary group. They are 
more involved with the system and may have a higher level of 
satisfaction, but not significant. 

It is noteworthy to examine the significant positive 
level of satisfaction towards the local MID staff and 
services. Although there was a statistically significant 
lower difference, the physicians' level of satisfaction for 
this factor was positive, the only factor to be scored 
positive by the physicians. The administrative and ancillary 
groups' satisfaction index indicated that they were very 
satisfied with the services provided by the local MID staff. 
Common sense would suggest that the level of satisfaction is 
directly related to effective communication ( Conrath and 
Mignen, 1990). Thus, it can be reasonable to assume that the 
MID staff is effectively communicating with the ancillary and 
administrative group and, to a lesser degree, the physicians. 

Bailey and Pearson (1983) listed contractor services 
and time required for system changes as the two most frequent 
elements for causing dissatisfaction among users. The outside 
services rendered by DMSSC were viewed unfavorably by the work 
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groups. DMSSC oversees 166 sites DoD wide. Each site has an 
AQCESS Customer Support Representative who is responsible for 
providing direct support of that user. A site has several 
means of reporting their problems; E-mail, System Change 
Requests (SCRs), work shops etc. (NMDSC, 1991) Any requests 
for changes to a system or its software, unless the change is 
critical, will be a lengthy process. Since most of the users 
are not familiar with the centralized process or DMSSC, it is 
only reasonable to expect the low level of satisfaction for 
this factor. 

It appears that the users will be satisfied if the 
output information is accurate, comprehensive, reliable and 
relevant, the users' will be satisfied. The work group was 
slightly satisfied with the factor regarding the quality of 
output information delivered by the AQCESS system. 

Powers and Dickson (1973) argues that no matter how 
effective a system might be, if it is not perceived to be 
satisfactory, it will be underutilized. AQCESS is training 
intensive and is not designed for the computer novice. 
Consequently, some of the features of the system are 
underutilized. (NMDSC, 1991) If that is the case, then a low 
level of satisfaction will be displayed by the users. The 
administrative and ancillary groups displayed a nearly neutral 
satisfaction index for the knowledge and involvement factor. 
The satisfaction index was negative for the physicians. The 
administrative group exhibited a satisfaction level which 
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showed a statistically significant higher difference than the 
other two groups. The administrative group can be considered 
slightly satisfied with this factor. 

5. Satisfaction versus Time of System Use 

In a study conducted by Igbaria and Nachman (1990), a 
significant relationship was found between user satisfaction 
and system utilization. This study showed an increase in 
overall satisfaction as time of system use increased. Then, 
the level of satisfaction decreases after extended use 
(greater than 11 months) of the system. 

Initially, new personnel are preoccupied with learning 
the AQCESS system in order to learn their job. Once they have 
become comfortable and familiar with the AQCESS system, the 
users are more concerned with doing their job. Consequently, 
the users become more intolerant to system down time or slow 
response time. Also, as their knowledge of and experience 
with the system grows, they can more easily comprehend the 
reasons for system problems and only wonder why the experts 
can not fix them. Thus, their level of satisfaction for the 
system has decreased with time. 

C. FORT ORD OVERALL SATISFACTION 

A comparison of the satisfaction levels of the respondents 
from the Silas B. Hays Hospital will provide a baseline 
measure for future evaluations of the system. 
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1 . Combined Groups 



Fort Ord's average satisfaction index is 10.47. By 
referring to the index ranges of -39 to 39, the respondents' 
average overall satisfaction is classified as being only 
slightly satisfied. 

2. Comparison of Work Groups 

There were two work groups identified in this study, 
administrative and ancillary, instead of three because no Fort 
Ord physicians responded to the survey. (A spokesman stated 
that the physicians did not interact with the system. ) There 
was no statistically significant difference between the work 
groups. However, the ancillary group was more satisfied than 
the administrative group. One problem with this assessment is 
that the ancillary group's surveyed population only composed 
approximately three and one half percent of the total 
respondent population as opposed to the administrative group's 
overwhelming 96.5 percent. Therefore, the ancillary group’s 
results may not be an accurate representation of the actual 
group population as a whole. 

Both administrative and ancillary personnel had very 
low regards for the timeliness of outside services and 
modifications to the system, as well as the amount of training 
they received. However, their collective satisfaction with 
the local IMD staff was considerably higher. The users 
appeared to recognize that the local staff was both willing 
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and capable of performing services which were not governed by 
higher authority. 

3. Significant Individual Question Differences and 

Grouped factors 

There was no statistically significant difference 
between the work groups for any of the responses to the 
individual questions or grouped factors. The ancillary group 
generally appeared to be more satisfied in the individual 
question areas as well as the four grouped factors. However, 
as suggested earlier, this may be attributed to the ancillary 
group's size. 

4. Satisfaction Versus Time of System Use 

The ancillary group only had users with six to 11 
months of system use, however, their level of satisfaction was 
relatively high. The administrative group's level of 
satisfaction actually increased as the respondents' time of 
system use increased. This trend is consistent with that 
which was predicted in chapter VII. Many respondents equated 
experience with the system to being able to efficiently and 
effectively manipulate the system and further translated that 
to "job satisfaction". 

D. PENSACOLA AND FORT ORD OVERALL SATISFACTION 

A comparison of the satisfaction levels of the respondents 
from the both systems will further aid in future evaluations. 
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1 . Combined Groups 

Combining both Pensacola and Fort Ord ' s work groups 
produces an average satisfaction index of 9.64. The results 
indicated that there was a low level of satisfaction among all 
AQCESS respondents . 

2 . Comparison of Work Groups 

The work groups identified in this study, included 
both Pensacola's and Fort Ord's administrative and ancillary 
groups. The administrative group displayed the highest level 
of satisfaction, followed by the ancillary group. It has been 
established that the administrative group's use of the 
information is less critical than that of the ancillary group. 
As a result, they are less frustrated with slow response and 
down time of the system. 

3. Significant Individual Question Differences and 

Grouped factors 

The combined work groups at Pensacola showed a 
statistically significant higher difference than Fort Ord for 
question 2. It would appear that neither site is at all 
satisfied with the way in which outside contractors or DMSSC 
process requests for changes to the system. Some users 
expressed dissatisfaction over not having their requests 
honored in a timely manner. Others understood and agreed with 
the process involved in implementing changes. After all, if 
all requests were acknowledged and implemented, there would 
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probably be a number of unique AQCESS systems throughout DoD. 
There was no statistically significant differences regarding 
the grouped factors. 

E. FORT ORD, PENSACOLA AND CHARLESTON OVERALL SATISFACTION 
Current plans for AQCESS include many new initiatives. 
AQCESS 6.0 was tested and accepted by representatives from the 
three services in March, 1991. Expanded functions have been 
incorporated into AQCESS 7.0 which include scrolling help 
windows with "point and shoot", data entry from help tables 
and Ad Hoc Reporting. Beta testing is expected to begin the 
during the first quarter of fiscal year 1992. DoD funding has 
been established to support hardware upgrades at selected 
sites in fiscal years 1992 and 1993. Despite all these 
upgrades to the AQCESS system, scheduling in September and 
December of 1992 the AQCESS systems at Pensacola and Fort Ord 
will be replaced by the CHCS system. (NMDSC, 1991) 

A comparison of the satisfaction levels of the respondents 
from the combined AQCESS systems and the CHCS system will aid 
in future evaluations. 

1 . Combined Groups 

The survey average satisfaction index is 9.64. 
Charleston is more satisfied with their system than Fort Ord 
or Pensacola. Pensacola shows a less than average level of 
satisfaction. This may be attributed to the fact that the 
Pensacola physicians displayed a very low level of 
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satisfaction for the AQCESS system. The differences in the 
levels of satisfaction were not statistically significant. 

2. Significant Individual Question Differences and 

Grouped factors 

This section will only discuss those questions that 
were statistically significant. 

a. Processing of Requests for Changes to the Existing 
System 

All sites responded negatively. Charleston is less 
satisfied than the other sites and happens to be a beta test 
site for the CHCS system. Software support for this site is 
furnished by a civilian contractor. The CHCS system is a 
relatively new system under development, whereas the AQCESS 
system has been operational since 1984. The developers and 
service personnel of the AQCESS system have had the 
opportunity to identify and correct many software problems. 
The AQCESS systems' MID/IMD personnel are less concerned with 
system implementation and more concerned with system 
maintenance, so there are fewer requests for changes. 
Therefore, the users of the AQCESS system may be less 
frustrated. 

h. Degree of Training Provided to Users 

Charleston was more satisfied with their level of 
training than the other sites. Since Charleston was a test 
site and was in the implementation phase, the users' formal 
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training was more extensive than the other sites. Therefore, 
Charleston was expected to show a higher level of 
satisfaction. 

c- User's Feeling of Participation 

Charleston was more satisfied with their 
involvement with the CHCS system than the users of the AQCESS 
system. This can be expected, especially when comparing the 
physicians of both sites. The CHCS physicians use the system 
for direct patient care. In fact, the CHCS physicains have a 
terminal located in their office. Whereas, the AQCESS 
physicians' only legitimate way of interfacing with the system 
is by reviewing statistics of their performance compiled by 
the Quality Assurance personnel. 

d. Attitude of the Information Management Staff 
Charleston was more satisfied with the Information 

Management staff. Since they are a test site, the MID staff 
provided more recent and extensive training. The 
communication between the CHCS users and their staff was 
expected to be more satisfying. 

e. Accuracy of Output Information 

The accuracy of output information for the CHCS 
physicians, in many instances, is directly related to the 
information that they input. The AQCESS physicians have 
little or no control over the input information. Therefore, 
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it is reasonable to assume that the level of satisaction for 



CHCS users will be higher. 

f. Factor A (Local MID/IMD Staff and Services) 

Charleston shows a higher level of satisfaction. 
The CHCS users' perception of the local MID is considered to 
be quite satisfactory. There does not appear to be any 
communication problems between the CHCS users and the MID 
staff. The AQCESS users appear to be satisfied with the level 
of service from their MID/IMD staff, but to a lesser degree. 
The higher level of satisfaction for the CHCS group can be 
attributed to the increased communication required between 
them and their MID staff while undergoing testing of the 
system. 

F. COMBINED CHCS AND AQCESS OVERALL SATISFACTION 

The work groups from the AQCESS and the CHCS systems have 
been combined to pinpoint problems of a particular work group. 
The following sections will analyze the responses from the 
combined work groups. 

1. Overall satisfaction 

The overall satisfaction index was 10.14. This places 
the overall satisfaction index in the lower third of the 
scale. This score should be used as a baseline measure for 
follow-on analysis of these hospital information systems. 

The ancillary and administrative groups, with 
comparable satisfaction indices, appear to be slightly 
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satisfied with their information systems. The physicians, 
with a near neutral satisfaction index, can be considered to 
be unsatisfied with the system. 

2. Significant Individual Questions 

The following section will only address those 
questions with statistically significant differences among the 
work groups. The individual questions will be combined into 
grouped factors and discussed in the next section. 

a. Process±ng of Requests for Changes to Existing 

Systems 

The administrative group shows a higher level of 
satisfaction. However the results show a neutral satisfaction 
index for the administrative group and negative satisfaction 
indices for the ancillary group and physicians. This implies 
that the work groups are dissatisfied with the contracted 
services, which is consistent with previous analysis. 

b. Degree of Training Provided to Users 

The physicians consistently demonstrated a lower 
level of satisfaction for training in all the previous 
comparisons. Clearly, this is an area for further study. 

c. User's Understanding of Systems 

The low level of satisfaction is evident among all 
work groups and, in particular, the physicians. This is 
consistent with previous findings that suggests users with 
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better computer background and training are more likely to be 
more satisfied with the system. 

d. Ussr' s Fsaling of Participation 

As previously noted, the users who are more 
involved with the system gain a better understanding of the 
system. As a result, are more satisfied. In the case of the 
AQCESS physicians, they are rarely involved with the system. 
Contrarily, the CHCS physicians with terminals in their 
offices, may find the system to be a distraction from their 
daily routine. Thus, the physicians' negative satisfaction 
index indicates their dissatisfaction for their current level 
of participation. 

e. Attitude of the Information Management Staff 

For this item, the physicians indicated 
satisfaction with the system, even though they displayed a 
lower level than the other groups . This is the highest 
satisfaction index recorded by the physicians. The 

administrative and ancillary groups were highly satisfied with 
the attitude of their MID/IMD staffs. 

f. Reliability of Output Information 

The physicians are not satisfied with the 
reliability of output information. This can be attributed to 
their roles and involvement with their systems . The 
administrative and ancillary personnel are only slightly 
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satisfied. If the information is inconsistent, the users will 
have a low level of satisfaction. 

g. Relevancy of Output Information ( to Intended 
Function 

The physicians perceived the output information as 
not being what they required. The administrative and 
ancillary groups were reasonably satisfied. 

h. Time Required for New Systems Development 
Contractor services and time required for system 

changes are viewed unfavorably by the work groups. Responses 
to requests for changes to information systems that are 
controlled by a central management ( DMSSC ) are more likely to 
be slower than responses for an information system that has a 
decentralized management. 

1. Completeness of the Output information 

All groups show a positive satisfaction index. 
However, consistent with previous analysis, the physicians 
perceived the information to be incomplete. If the 
information is perceived to be inaccurate, irrelevant and 
inconsistent, users will not be satisfied with the output 
information. 

3. Comparison of Grouped Factors 

The physicians are significantly less satisfied for 
all factors. That could be, for the most part, due to 
combining the CHCS physician responses with those of the 
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AQCESS physicians. Although the sample populations for the 
two physician groups differ by only one, the extremely 
negative scores reported by the Pensacola group significantly 
influenced the overall low scores of the combined physician 
group. Nevertheless, the physicians tend to show a positive 
level of satisfaction for the MID/IMD staff and services along 
with the administrative and ancillary groups. 

The analysis of the responses form the work groups for 
contractor services and time required for system changes are 
consistent with other analysis. The administrative and 
physician groups view the contractor services negatively, 
while the ancillary group is neutral. This was the most 
frequent cause of dissatisfaction for the users. 

The administrative and ancillary groups were slightly 
satisfied with the quality of output delivered by the system. 
The physicians, however, were dissatisfied. 

The physicians were dissatisfied with knowledge of and 
involvement with the system. If users are lacking in 
training, understanding, and experience with the system, they 
are more likely to be less satisfied. 
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XII. CONCLUSIONS AND RECOMMENDATIONS 



A. MEASURING USER INFORMATION SATISFACTION 

Today, there is a greater need to understand the user's 
point of view. As the availability of hardware and software 
continues to grow, it becomes more difficult to remain current 
with the pace of technological changes. Because of personal 
computing, users are far more knowledgeable. They are less 
tolerant of complex time-consuming processes and procedures, 
delays due to backlogs and shifting priorities and inadequate 
procedures and information on available resources. (Conrad and 
Mignen, 1990, Cash et al, 1988, Mensching and Adams, 1991) 
Conrad and Mignen (1988) argue that the focus should not be on 
the cure for user dissatisfaction, but to develop the means to 
identify problems before they occur. This is almost 
impossible without some means of gauging users' perceptions. 
The user information satisfaction survey questionnaire 
provides the means for gauging users' perceptions. The survey 
questionnaire is easily and quickly administered and provides 
a standard measure for comparisons of scores across 
departments, systems, users, organizations, and industries. 
This survey is not a conclusive evaluation of the AQCESS and 
CHCS systems, but coupled with further investigation, can be 
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a powerful tool in the analysis and interpretation of the 
causes of user dissatisfaction. 

B. SATISFACTION CHARACTERISTICS OF AQCESS 

Overall satisfaction is a generalization of all the 
characteristics that effect the satisfaction of the user. 
Along with overall satisfaction, the study specifically looked 
at the four factors that make up satisfaction. 

1 . Overall Satisfaction 

The survey results indicate the users are slightly 
satisfied with AQCESS at Silas B. Hays Army Hospital, Fort 
Ord, and by Naval Hospital, Pensacola. The physicians' 
responses were consistently scored low across CHCS and AQCESS. 
The lack of Fort Ord physician responses to the survey may 
have caused a slightly inflated overall satisfaction score for 
them. The overall satisfaction scores across organizations 
and systems did not differ significantly. 

2. Areas of Satisfaction 

All three work groups (to a lessor extent the 
physicians) rated the local MID/IMD staff and services as 
being satisfactory. However, the CHCS users' satisfaction 
index is twice that of the AQCESS users'. This suggests that 
the local AQCESS MID/IMD personnel are not communicating and 
interacting with their users as effectively as the CHCS MID 
personnel are with their users. CHCS ' s success in this area 
can be attributed to the emphasis on training the users during 
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the implementation phase of the CHCS system. The AQCESS 
systems' MID/IMD personnel, far beyond the implementation 
phase, have no need to intensify the training of their users. 
However, they do maintain a solid steady training program that 
includes both formal and on-the-job training. 

In many areas, the AQCESS systems' administrative 
groups displayed higher satisfaction than any other group. 
The ancillary groups, whose interaction with the system is 
more essential (regarding direct patient care), were more 
dissatisfied because of the system's response time. 

3. Areas of Dissatisfaction 

Armed with a set of problem areas, the MTD/IMD staff 
and functional managers of the AQCESS systems can explore the 
underlining causes through: (1) interviews and (2) by 
examining the development and operating procedures of 
particular user groups within their systems. Investigation 
into areas of dissatisfaction can provide context, history and 
insight for possible corrective measures. 

Since the physicians' satisfaction indices were low, 
across the organizations, it seems to be a characteristic of 
physicians to display a low level of satisfaction for hospital 
information systems. The limited involvement of the Pensacola 
physicians was a major factor for their overall low level of 
satisfaction. 
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The respondents were most dissatified with the amount 
of time it took for new system development and changes to 
occur. AQCESS is a centrally managed information system, 
therefore it requires a relatively long time to process 
requests for changes. 

4. Satisfaction with Time of System Use 

Regression analysis revealed no correlation between 
satisfaction and time of system use. The Fort Ord 
administrative group displayed increased satisfaction with 
increased time of use. All Pensacola groups, with greater 
than 11 months of experience, were less satisfied. 

C . RECOMMENDATIONS 

The short- form questionnaire developed by Baroudi and 
Orlikowski (1988) is an effective means to measure user 
information satisfaction. This survey instrument is an 
appropriate instrument to document user satisfaction within 
Medical Treatment Facilities as well as documenting changes in 
user satisfaction at Naval Hospitals, Pensacola and Silas B. 
Hays (Army) Hospital at a later time. 

This type of survey, to prevent biased results, is best 
conducted by a researcher who is not experienced with the 
system. Thus, the interpretation of the results are more 
likely to reflect impartiality. It is recommended that the 
researcher arranges with a member of the organization to serve 
as a point of contact. The contact distributes and collects 
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the questionnaires for the researcher. It should be 
emphasized prior to the survey that the questionnaire elicits 
responses reflective of the present conditions and not the 
past conditions and experiences with information systems or 
MID/IMD staffs. In employing the user satisfaction 

questionnaire, it was found that some individuals were 
uncertain as to the exact meaning of certain questions. If 
the lack of clarity is likely to be a problem, it is 
recommended that full explanations of the scales be included. 
Additionally, direct contact with individuals for interview by 
the researcher is beneficial for adding context to history. 
Additionally, the following is recommended prior to conducting 
the interview: 

1 . Structure the interviews around known problem areas to 
avoid "orienting" time spent in searching for real 
issues. 

2. Avoid focusing on the highly specific, idiosyncratic 
complaints of individual users that are not of general 
concern. 

3. Reduce the number of interviews needed to obtain deeper 
undertsanding of the problem areas. 

To obtain a deeper understanding of the issues by adding 
context to history, it is also recommended that the following 
items be further investigated: 

1. The administrative group's overall higher satisfaction 
over the other work groups. 

2. The physicians' lowered perception of satisfaction in 
almost all areas. 

3. The reasons for the respondents having negative 
perceptions of satisfaction with contractor's services. 
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The user satisfaction questionnaire should be used as a 
standard measuring tool. This tool will allow both the 
researcher and the practitioner to utilize a readily available 
instrument, thus, avoiding the process of developing a new 
measure each time an assessment of user satisfaction is 
required. A follow on study should be conducted at Silas B, 
Hays Hospital, Fort Ord and Naval Hospital, Pensacola, using 
the results of this study as a baseline comparison. Finally, 
the results of this study should be used for measuring user 
satisfaction at other DoD hospital sites. 
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APPENDIX A 



Part A: General Information 

1. Hospital Division/Department: (Check one) 

General Administration 

Nursing Administration 

Dietary 

Emergency 

Laboratory 

Medical Clinic 

Inpatient Nursing 

Pharmacy 

Radiology 

Other (Specify): 

2. Job Description: (Check one) 

Administration 

Medic 

Technician 

Nurse 

Pharmacist 

Physician 

Physician Assistant 

Other (Specify): 

3. Highest Level of Education: (Check one) 

High School Graduate 

Some College 

Associate ' s Degree 

Bachelor's Degree 

Some Graduate Work 

Master's Degree 

Doctoral Degree 

Medical Degree 

Other (Specify): 

4 . Age : 

5. Gender: Male Female 

6. Length of time (in months) you have used AQCESS: 

7. Have you used other computer systems before ? Yes No 

8. If your answer was Yes to question 7, was it a health care 

information system ? Yes No 
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Part B; User Satisfaction Questionnaire 

This section conveys your personal feelings concerning the use 
of the Automated Quality of Care Support System at Naval Hospital, 
Pensacola / Silas B. Hays Hospital (Fort Ord). Please do not 
attempt to analyze the questions. Remember, there are no right oi 
wrong answers. 

Please follow these instructions: 

1 . Check each scale in the position that describes your 

evaluation or answer. Example: Inventor of the traffic light 

genius : 3^ ; : : 2 _ : : : 3 : incompetent 

You would check an "X" in the left-hand side of the scale 
(marked as "1" in the above example) if you thought the 
inventor was a genius. Similarly, you would mark the 
mid-point of the scale ("2" in the above example) if you 
thought the inventor was neither a total genius nor totally 
incompetent and the right-hand side (point "3" in the above 
scale) if you thought the inventor was incompetent. The other 
positions can be used to indicate varying degrees between 
"1" and "2" or "2" and "3". 



2. Check in the space, not between spaces (be sure to 
check only one position per scale). 

(Correct way -♦ : x : Incorrect way -> : x : ) 

3. Check both of the scales after each question. 

4. Work rapidly, do not omit any questions and rely on your 
first impressions. 



ANSWERS BASED ON YOUR OWN OPINIONS 

1. Relationship with the Management Information Department (MID) 
Information Management Division (IMD) staff 

dissonant : : : : : : : : harmonious 

bad : : : : : : : : good 

2. Processing of requests for changes to existing systems 

fast : : : : : : : : slow 

untimely : : : : : : : : timely 

3. Degree of training provided to users 

complete : : : : : : : : incomplete 

low : : : : : : : : high 

4 . User ' s understanding of systems 

insufficient : : : : : : : : sufficient 

complete : : : : : : : : incomplete 
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5. User's feeling of participation 

positive : : : : : : : : negative 

insufficient sufficient 



6. Attitude of the Management Information Department / Information 
Management Division staff 

cooperative : : : : : : : : belligerent 

negative : : : : : : : : positive 



7. Reliability of output information 

high : : : : : : : : low 

superior : : : : : : : : inferior 



8. Relevancy of output information (to intended function) 

useful : : : : : : : : useless 

relevant irrelevant 



9. Accuracy of output information 

inaccurate : ; : : ; : : : accurate 

low : ; : : : : : : high 

10. Precision of output information 

low : ; : : : : : : high 

definite uncertain 



11. Communication with the Management Information Department / 
Information Management Division staff 

dissonant : : : : : : : : harmonious 

destructive : : : : : : : : productive 



12. Time required for new systems development 

unreasonable : : : : : : : : reasonable 

acceptable : : : : : : J J unacceptable 

13. Completeness of the output information 

sufficient : : : : : : : : insufficient 

adequate : : : : : : : : inadequate 



Thank you again for your cooperation 
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APPENDIX B 



Automated Quality of Care Evaluation 
Support System Survey 



This survey is part of a study of the Automated Quality of Car 
Evaluation Support System (AQCESS) and its effectiveness in 
military hospital settings. The purpose of the study is to obtain 
information regarding your perceptions or views of how well AQCESS 
functions in your specific area of use. 

Just a few minutes are required to fill out general informatio: 
along with a 13 question survey. Your identity will remain 
confidential. The data gathered through the survey will provide 
valuable insight into the system's strengths and/or weaknesses as 
well as assist in future research, development and training. 

The success of this survey depends on my receiving as many 
completed surveys as possible from users in all hospital areas. 
Your participation lends an important contribution towards this 
end. There are no right or wrong answers, only your candid 
response to each question. Please do not omit any of the question: 
in either section of the survey. 

Please return your completed survey to LCDR Neeley in the 
Management Information Department / Mr. G. Scott (HSXT-IMD) in the 
envelope provided. Thank you for your cooperation. 



James R. Booth, LCDR, USN / John L. Bryant, Jr. , LT, US 

Naval Postgraduate School 
SMC 1073 / SMC 2604 

Monterey, CA 93940-4831 
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APPENDIX C-1 



NAVAL HOSPITAL PENSACOLA 

COMBINED RESULTS FOR PHYSICIANS VS ANCILLARY VS 
ADMINISTRATIVE 



TOTAL SATISFACTION 


AVERAGE 


INDEX 








PHYSICIAN 




-0.3899 


-4.15909 








ADMIN 




1.42659 


12.83929 








ANCILLARY 




0.72624 


11.49265 








GROUP FACTORS 


A 


A 


B 


C 








(MID) 


(CON) 








PHYSICIAN 


SAT 


0.64394 


-0.76136 


-0.4955 


-1.01136 


ANCILLARY 


SAT 


1.41176 


0.003676 


0.70882 


0.376838 


ADMIN SAT 




1.7123 


0.309524 


1.00357 


0.59375 


PHYSICIAN 


STDEV 


1.43618 


1.538883 


1.53888 


1.595408 


ANCILLARY 


STDEV 


1.52979 


1.694494 


1.73813 


1.816868 




ADMIN STDEV 


1.44426 


1.76929 


1.69909 


1.828224 




INDIVIDUAL QUESTION 


RESPONSES 








QUESTION 




1 


2 


3 


4 


5 


PHYSICIAN 




0.63636 


-1.04545 


-1 


-0.75 


-1.25 


ANCILLARY 




1.5 


-0.09504 


0.13971 


0.69117 


0.82353 


ADMIN 




1.72024 


0.309524 


0.19048 


0.95833 


0.91667 


STAN DEV 




1 


2 


3 


4 


5 


PHYSICIAN 




1.58245 


1.445283 


1.43019 


1.79804 


1.63936 


ANCILLARY 




1.53872 


1.800855 


1.85968 


1.75530 


1.67104 


ADMIN 




1.40121 


1.883362 


1.81578 


1.80394 


1 . 67053 


QUESTION 




6 


7 


8 


9 


10 


PHYSICIAN 




0.5 


-0.61364 


-0.6591 


-0.4772 


-0.3182 


ANCILLARY 




1.39706 


0.941176 


0.91176 


0.61029 


0.35294 


ADMIN 




1.88095 


0.952381 


1.30357 


0.97023 


0.68452 


STAN DEV 




6 


7 


8 


9 


10 


PHYSICIAN 




1.58831 


1.555283 


1.46075 


1.42204 


1.60642 


ANCILLARY 




1.57312 


1.670526 


1.67363 


1.73702 


1.68265 


ADMIN 




1.41762 


1.76544 


1.52268 


1.72318 


1.70133 


QUESTION 




11 


12 


13 






PHYSICIAN 




0.79545 


-0.47727 


-0.4091 






ANCILLARY 




1.33824 


0.154412 


0.72794 






ADMIN 




1.53571 


0.213178 


1.10714 
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STAN DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 



11 12 
1.057 1.097283 
1.4714 1.566556 
1.49161 1.647337 



13 

.61412 

.85293 

.85293 
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APPENDIX C-2 



NAVAL HOSPITAL PENSACOLA 

PHYSICIANS VS THE REST OF THE SURVEY GROUP 



GENERAL 

PHYSICIAN 

STDEV 

NON-PHYSICIAN 

STDEV 

GROUP FACTORS 

PHYSICIAN GROUP 
NON-PHYSICIAN 
PHYSICIAN STDEV 
NON-PHYSICIAN STDEV 

QUESTION 

PHYSICIAN SAT 
PHYSICIAN STDEV 
NON-PHYSICIAN SAT 
NON-PHYSICIAN STD 

QUESTION 

PHYSICIAN SAT 
PHYSICIAN STDEV 
NON-PHYSICIAN SAT 
NON-PHYSICIAN STD 

QUESTION 

PHYSICIAN SAT 
PHYSICIAN STDEV 
NON-PHYSICIAN SAT 
NON-PHYSICIAN STD 



AVERAGE 


INDEX 




-0.3899 


-4.1591 




1.62264 


9.4067 




0.8707 


13.7796 




1.75216 


20.3155 




A 


A 


B 


(MID) 


(CON) 




0.64394 


-0.76140 


-0.49550 


1.57785 


0.17270 


0.87171 


1.43618 


1.31421 


0.28359 


1.49064 


1.74287 


1.72291 


1 


2 


3 


0.63636 


-1.04550 • 


-1.0000 -1 


1.58245 


1.44528 


1.4301 


1.62171 


0.10526 


0.1677 i 


1.46841 


1.86081 


1.8357 


6 


7 


8 


0.50000 


-1.0455 ■ 


-1.0000 -1 


1.58831 


1.55528 


1.4607 


1.66447 


0.94737 


1.1282 1 


1.50850 


1.72363 


1.6038 


11 


12 


13 


0.79545 


-0.47730 • 


-0.4091 


1.05700 


1.09728 


1 . 61412 


1.44737 


0.24013 


0.8914 


1.48585 


1.61354 


l^ 78611 



C 

-1.0114 

0.49671 

0.31756 

1.82634 

4 5 

.7500 -1.2500 
.7980 1.63936 
.8388 0.87500 
.7872 1.67140 

9 10 

.6136 -0.3182 
.4220 1.49033 
.8092 0.53618 
.7386 1.70101 
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APPENDIX C-3 



PHYSICIANS VS THE REST OF THE SURVEY GROUP 



CALCULATED 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 
13 



T- VALUE 
2.91074* 
2.78081* 
2.86054* 
3.89161* 
5.73435* 
3.34561* 
5.13898* 
5.89105* 
3.60145* 
2.24833* 
2.00319* 
2.02872* 
3.22161* 



FACTOR 

A(MID) 

A(CON) 

B 

C 



2.77688* 

2.41541* 

3.71987* 

3.91692* 



OVERALL 4.07910* 



* SIGNIFICANT AT ALPHA = .05 
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APPENDIX C-4 



NAVAL HOSPITAL PENSACOLA 



ANCILLARY VS THE REST OF THE SURVEY GROUP 



GENERAL 


AVERAGE 


INDEX 








ANCILLARY 


0.72624 


11.49265 








NON-ANCILLARY 


0.70174 


9.311321 








ANCILLARY STDEV 


1.75436 


21.45832 








NON-ANCILLARY STDEV 


1.80618 


14.53046 








GROUP FACTORS 


A 


A 


B 


C 






(MID) 


(CON) 








ANCILLARY 


1.41176 


0.003676 


0.70882 


0.376838 


NON-ANCILLARY 


1.49057 


0.087264 


0.69245 


0.175393 


ANCILLARY STDEV 


1.52979 


1.694494 


1.73813 


1.788388 


NON-ANCILLARY STDEV 


1.50625 


1.740047 


1.76784 


1.897845 




QUESTIONS 


1 


2 


3 


4 


5 


ANCILLARY 


1.5 


-0.14706 


0.13971 


0.69117 


0.82353 


NON-ANCILLARY 


1.49528 


0.028302 


-0.0566 


0.60377 


0.46698 


ANCILLARY STDEV 


1.53872 


1.800855 


1.85968 


1.75530 


1.67104 


NON-ANCILLARY STDEV 


1.50627 


1.883181 


1.80842 


1.93126 


1.88185 


QUESTIONS 


6 


7 


8 


9 


10 


ANCILLARY 


1.39706 


0.941176 


0.91176 


0.61029 


0.82353 


NON-ANCILLARY 


1.59434 


0.627358 


0.89623 


0.66981 


0.47642 


ANCILLARY STDEV 


1.57312 


1.670526 


1.67363 


1.73702 


1.68265 


NON-ANCILLARY STDEV 


1.55877 


1.837196 


1.70697 


1.76561 


1.73053 


QUESTIONS 


11 


12 


13 






ANCILLARY 


1.33471 


0.154412 


0.72794 






NON-ANCILLARY 


1.38208 


0.146226 


0.79245 






ANCILLARY STDEV 


1.4714 


1.566556 


1.85293 






NON-ANCILLARY STDEV 


1.444 


1.581835 


1.79998 
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APPENDIX C-5 



ANCILLARY vs THE REST OF THE SURVEY GROUP 

CALCULATED T-VALUE 

1 0.00000 

2 0.62229 

3 -0.69054 

4 -0.30014 

5 -1.25072 

6 0.78195 

7 -1.12362 

8 0.22088 

9 0.21962 

10 -1.27920 

11 0.22167 

12 0.23056 

13 0.21223 

FACTOR 

A(MID) 0.34149 
A(CON) 0.31046 
B -0.03660 

C -0.69144 

OVERALL -1.50880 



* SIGNIFICANT AT ALPHA = .05 
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APPENDIX C-6 



NAVAL HOSPITAL PENSACOLA 

ADMINISTRATIVE VS THE REST OF THE SURVEY GROUP 



GENERAL 


AVERAGE 


ADMIN SAT 


0.98764 


NON-ADMIN SAT 


0.45342 


ADMIN STDEV 


1.74163 


NON-ADMIN STDEV 


1.78861 


GROUP FACTORS 


A 




(MID) 


ADMIN 


1.7123 


NON-ADMIN 


1.22407 


ADMIN STDEV 


1.44426 


NON-ADMIN STDEV 


1.54314 


QUESTIONS 


1 


ADMIN SAT 


1,72024 


NON-ADMIN SAT 


1.28889 


ADMIN STDEV 


1.40121 


NON-ADMIN STDEV 


1.59335 


QUESTIONS 


6 


ADMIN SAT 


1.88095 


NON-ADMIN SAT 


1.17778 


ADMIN STDEV 


1.41762 


NON-ADMIN STDEV 


1.62329 


QUESTIONS 


11 


ADMIN SAT 


1.53571 


NON-ADMIN SAT 


1.20556 


ADMIN STDEV 


1.49161 


NON-ADMIN STDEV 


1.40118 



INDEX 

12.83929 

7.666667 

13.53834 

20.36609 








A 

(CON) 


B 


C 




0.309524 

-0.18333 

1.76929 

1.643083 


0.98148 

0.41444 

1.69909 

1.769 


0.59375 

0.0375 

1.828224 

1.863385 


2 


3 


4 


5 


0.309524 

-0.36667 

1.883362 

1.763519 


0.19048 

-0.1389 

1.81578 

1.8311 


0.95833 

0.33888 

1.80394 

1.87131 


0.91667 

0.31667 

1,67053 

1.88702 


7 


8 


9 


10 


0.952381 

0.561111 

1.76544 

1.773772 


1.30357 

0.52778 

1.52268 

1.75888 


0.97023 

0.34444 

1.72318 

1.72987 


0.68452 

0.18889 

1.70133 

1.68914 



12 


13 


0.309524 


1.10714 


0.096939 


0.45 


1.647337 


1.71143 


1,490712 


1.86272 
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APPENDIX C-7 



ADMINISTRATIVE vs THE REST OF THE SURVEY GROUP 
CALCULATED T-VALUE 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 
13 


-1.88791 
-2.46426 * 
-1.19177 
-2.22523 * 
-2.21289 * 
-3.02213 * 
-1.45634 
-3.07875 * 
-2.40038 * 
-1.90959 
-1.50105 
-0.92410 
-2.43150 * 


FACTOR 

A(MID) 

A(CON) 

B 

C 


-2.16394 * 
-1.89548 * 
-2.16356 * 
-0.07198 * 


OVERALL 


-1.45925 



* SIGNIFICANT AT ALPHA = .05 
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APPENDIX C-8 



NAVAL HOSPITAL PENSACOLA 



ANCILLARY VS ADMINISTRATIVE GROUP 



GENERAL 
ANCILLARY SAT 
ADMIN SAT 
ANCILLARY STDEV 
ADMIN STDEV 



AVERAGE INDEX 
0.72624 13.73530 

0.98764 13.81550 

1.75436 25.13573 

1.74163 15.34150 



GROUP FACTORS 

ANCILLARY SAT 
ADMIN SAT 
ANCILLARY STDEV 
ADMIN STDEV 



A A B C 
(MID) (CON) 

1.41176 0.003676 0.70882 0.37684 
1.71230 0.30952 0.98148 0.59375 
1.52979 1.694494 1.73813 1.78839 
1.44426 1.76929 1.69909 1.82822 



QUESTIONS 
ANCILLARY SAT 
ADMIN SAT 
ANCILLARY STDEV 
ADMIN STDEV 



1 

1.50000 

1.72024 

1.53872 

1.40121 



- 0.14710 

0.30952 

1.80086 

1.88336 



3 

0.13971 

0.19048 

1.85968 

1.81578 



4 

0.69118 

0.95833 

1.75531 

1.80394 



QUESTIONS 
ANCILLARY SAT 
ADMIN SAT 
ANCILLARY STDEV 
ADMIN STDEV 



6 

1.39706 

1.88095 

1.57312 

1.41762 



7 

0.94118 

0.95238 

1.67053 

1.76544 



8 

0.91176 

1.30357 

1.67363 

1.52268 



9 

0.61029 

0.97024 

1.73702 

1.72318 



QUESTIONS 
ANCILLARY SAT 
ADMIN SAT 
ANCILLARY STDEV 
ADMIN STDEV 



11 

1.33471 

1.53571 

1.47140 

1.49161 



12 

0.15441 

0.30952 

1.56656 

1.64734 



13 

0.72794 

1.10714 

1.85293 

1.71143 



5 

0.82353 

0.91667 

1.67104 

1.67053 



10 

0.82353 

0.68452 

1.68265 

1.70133 
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APPENDIX C-9 



ANCILLARY VS ADMINISTRATIVE GROUP 



CALCULATED T-VALUE 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 
13 



0.84816 

1.51933 

1.66100 

0.93099 

0.36708 

2.05345 

0.03501 

1.38279 

1.27973 

-0.50710 

0.81736 

0.56945 

1.31714 



FACTOR 

A(MID) 0.03318 
A(CON) 1.08134 
B 0.97679 
C 0.68113 



OVERALL 0.030183 

* SIGNIFICANT AT ALPHA = 



.05 
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APPENDIX C-10 



NAVAL HOSPITAL PENSACOLA 

SATISFACTION BASED ON TIME ON THE SYSTEM 



OVERALL 



# PEOPLE 

35 

34 

104 

174 



TIME OF 
USE 

1 TO 5 
6 TO 11 
>= 12 



AVG SAT 

11.94 

12.66 

7.58 



ADMINISTRATIVE 



# PEOPLE 

19 

17 

48 

84 

ANCILLARY 



# PEOPLE 

12 

13 

68 



PHYSICIANS 



# PEOPLE 

4 

4 

14 

22 



TIME OF 
USE 

1 TO 5 
6 TO 11 
>= 12 



TIME OF 
USE 

1 TO 5 
6 TO 11 
>= 12 



TIME OF 
USE 

1 TO 5 
6 TO 11 
>= 12 



AVG SAT 

15.16 

16.07 

10.99 



AVG SAT 

9.96 

13.71 

9.45 



AVG SAT 

10.07 

3.54 

6.36 
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APPENDIX D-1 



SILAS B. HAYS ARMY HOSPITAL 

COMBINED RESULTS FOR ADMINISTRATIVE AND ANCILLARY 



TOTAL SATISFACTION 


AVERAGE 


INDEX 






ADMIN 


0.79860 


10.38182 




STAN DEV 


1.76524 


15.80275 




ANCILLARY 


1 . 00000 


13.00000 




STAN DEV 


1.42774 


13.50000 




GROUP FACTORS 


A 


A 


B 


C 




(MID) 


(CON) 






ADMIN AVG 


1.27878 


0.05909 


0.94545 


0.34545 


ANCILLARY AVG 


1.54545 


0.37500 


0.90000 


0.87500 


ADMIN STDEV 


1.74456 


1.76356 


1.67894 


1.77449 


ANCILLARY STDEV 


1.83031 


2.34187 


1.17898 


1.72753 


COMBINED GROUP FACTORS 


A 


A 


B 


C 




(MID) 


(CON) 






AVERAGE 


1.28739 


0.07017 


0.94386 


0.36403 


STAN DEV 


1.73276 


1.78796 


1.66396 


1.77554 



INDIVIDUAL QUESTION RESPONSE 



QUESTION 


1 


2 


3 


4 


5 


ADMIN 


1.22727 


-0.31818 


0.118182 


0.90909 


0.67272 


ANCILLARY 


1 . 50000 


0 


0.750000 


1.50000 


1.25000 


STAN DEV 


1 


2 


3 


4 


5 


ADMIN 


1.72499 


1.73169 


1.73588 


1.56405 


1.79448 


ANCILLARY 


1 . 50000 


3.00000 


0.829156 


0.50000 


0.82915 


QUESTION 


6 


7 


8 


9 


10 


ADMIN 


1.15454 


0.95454 


1.20909 


0.91818 


0.80000 


ANCILLARY 


1.66666 


1.25000 


1 . 50000 


0 


1.00000 


STAN DEV 


6 


7 


8 


9 


10 


ADMIN 


1.82501 


1.691618 


1.65746 


1.70630 


1.58286 


ANCILLARY 


0.47140 


0.829156 


0.50000 


1.41421 


1.00000 


QUESTION 


11 


12 


13 






ADMIN 


1.45454 


0.43636 


0.84545 






ANCILLARY 


1 . 50000 


0.75000 


0.75000 






STAN DEV 


11 


12 


13 






ADMIN 


1.66589 


1.71372 


1.72250 






ANCILLARY 


1 . 50000 


1.29903 


1.29903 
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APPENDIX D-2 



ANCILLARY VS 


ADMINISTRATIVE 


CALCULATED T- 
1 
2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 
13 


-VALUE 

-0.22114 

-0.24072 

-0.50956 

-0.52982 

-0.45345 

-0.40038 

-0.24837 

-0.25320 

0.75420 

-0.17685 

-0.04192 

-0.25435 

0.07299 


FACTOR 

A(MID) 

A(CON) 

B 

C 


-0.16331 

-0.24566 

0.04106 

-0.40956 


OVERALL 


-0.23099 



* SIGNIFICANT AT ALPHA = .05 
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APPENDIX D-3 



SILAS B. HAYS ARMY HOSPITAL 
SATISFACTION BASED ON TIME OF USE 



OVERALL 



# PEOPLE 

12 

11 

M 

57 



TIME OF 
USE 

1 TO 5 
6 TO 11 
>= 12 



AVG SAT 

5.291667 

11.22727 

12.05882 



ADMINISTRATIVE 



# PEOPLE 

12 

9 

M 

55 



TIME OF 
USE 

1 TO 5 
6 TO 11 
>= 12 



AVG SAT 

5.291667 

10.83333 

12.05882 



ANCILLARY 



# PEOPLE 

0 

2 

0 

2 



TIME OF 
USE 

1 TO 5 
6 TO 11 
>= 12 



AVG SAT 

N/A 

13 

N/A 
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APPENDIX E-1 



NAVAL HOSPITAL PENSACOLA AND SILAS B. HAYS ARMY HOSPITAL FT. ORD 
COMBINED RESULTS 



OVERALL 


QUESTION 


INDEX 








AVERAGE 


0.7345987 


9.636364 








STAND DEV 


1.7789492 


14.83246 








TOTAL SATISFACTION 


QUESTION 


INDEX 








PHYSICIAN 


-0.38986 


-4.15909 








STAND DEV 


1.6226442 


9.406703 








ADMIN 


0.912839 


11.86691 








STAND DEV 


1.753449 


14.52653 








ANCILLARY 


0.7340659 


9.464789 








STAND DEV 


1.7464702 


14.38562 








GROUP FACTORS 


A 


A 


B 


C 






(MID) 


(CON) 








PHYSICIAN SAT 


0.0606061 


-0.76136 


-0.49545 


-1.01136 




ANCILLARY SAT 


1.4119048 


0.252381 


0.714286 


0.391071 




ADMIN SAT 


1.5407674 


0.210432 


0.980576 


0.495504 




PHYSICIAN STDEV 


1.4361806 


1.538883 


1.538883 


1.595408 




ANCILLARY STDEV 


1.524202 


1.717497 


1.724967 


1.816273 




ADMIN STDEV 


1.5842124 


1.771263 


1.691382 


1.81123 




OVERALL AVG 


1.4163059 


0.058442 


0.759307 


0.320346 




OVERALL STDEV 


1.573773 


1.739173 


1.717359 


1.845252 




INDIVIDUAL QUESTION 


RESPONSES 










QUESTION 


1 


2 


3 


4 


5 


PHYSICIAN 


0.6363636 


-1.04545 


-1 


-0.75 


-1.25 


ANCILLARY 


1.5 


-0.14286 


0.157143 


0.714286 


0.835714 


ADMIN 


1.5251799 


0.061151 


0.161871 


0.938849 


0.820144 


STAN DEV 


1 


2 


3 


4 


5 


PHYSICIAN 


1.582445 


1.445283 


1.430194 


1.798042 


1.63936 


ANCILLARY 


1.5376234 


1.846121 


1.841084 


1.737345 


1.654478 


ADMIN 


1.5562866 


1.850491 


1.784945 


1.713216 


1.724774 


QUESTION 


6 


7 


8 


9 


10 


PHYSICIAN 


0.5 


-0.61364 


-0.65909 


-0.47727 


-0.31818 


ANCILLARY 


1.3928571 


0.95 


0.928571 


0.592857 


0.371429 


ADMIN 


1.5935252 


0.953237 


1.266187 


0.94964 


0.730216 


STAN DEV 


6 


7 


8 


9 


10 


PHYSICIAN 


1.5883096 


1 . 555283 


1.460747 


1.422045 


1.606418 


ANCILLARY 


1.5569954 


1.653244 


1.654616 


1.731624 


1.670513 


ADMIN 


1.6304966 


1.736606 


1.684424 


1.716713 


1.656431 
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QUESTION 

PHYSICIAN 

ANCILLARY 

ADMIN 



11 12 
0.7954545 - 0.47727 
1.3428571 0.171429 
1.5035971 0.359712 



13 

- 0.40909 

0.728571 

1.003597 



STAN DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 



11 12 
1.0570015 1.097283 
1.4724684 1.562703 
1.5634039 1.67507 



13 

1.614116 

1.839421 

1.839421 
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APPENDIX E-2 



ACCESS FT ORD vs AQCESS PENSACOLA 



OVERALL 




QUESTION 


INDEX 






PENSACOLA 




0.71132 


9.02586 






FT ORD 




0.80567 


10.47368 






PENSACOLA 


STDEV 


1.78615 


14.93465 






FT ORD STDEV 


1.75489 


15.73504 






GROUP FACTORS 


A 


A 


B 


C 








(MID) 


(CON) 








PENSACOLA 




0.89719 


0.05452 


0.69885 


0.30582 




FT ORD 




1.28363 - 


-0.03955 


0.94386 


0.36403 




PENSACOLA 


STDEV 


1.51598 


1.72164 


1.76039 


1.86665 




FT ORD STDEV 


1.13162 


1.77987 


1.65140 


1.77555 




INDIVIDUAL 


QUESTION RESPONSES 








QUESTION 




1 


2 


3 


4 


5 


PENSACOLA 




1.49713 


-0.04011 


0.02012 


0.63793 


0.60632 


FT ORD 




1.23684 


-0.30702 


0.14035 


0.92998 


0.69299 


STAN DEV 




1 


2 


3 


4 


5 


PENSACOLA 




1.51903 


1.85706 


1.83113 


1.86497 


1.81078 


FT ORD 




1.71834 


1.79242 


1.71616 


1.54306 


1.77273 


QUESTION 




6 


7 


8 


9 


10 


PENSACOLA 




1.51724 


0.75000 


0.90210 


0.64655 


0.42816 


FT ORD 




1.15789 


0.96491 


1.21930 


0.88596 


0.80702 


STAN DEV 




6 


7 


8 


9 


10 


PENSACOLA 




1.56735 


1.78052 


1.69403 


1.75474 


1.71304 


FT ORD 




1.79951 


1.66980 


1.63170 


1.70530 


1.56652 


QUESTION 




11 


12 


13 






PENSACOLA 




1.36494 


0.14943 


0.76724 






FT ORD 




1.45614 


0.44444 


0.84211 






STAN DEV 




11 


12 


13 






PENSACOLA 




1.45492 


1.57589 


1.82113 






FT ORD 




1.66038 


1.65018 


1.70951 
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APPENDIX E-3 



AQCESS FT ORD vs AQCESS PENSACOLA 
CALCULATED T-VALUE 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 
13 


-1.65958 

0.92434 

-0.43705 

-1.06333 

-0.32762 

-1.34927 

-0.82097 

-1.17281 

-0.86718 

-0.47880 

-0.26139 

-1.17853 

0.37824 


FACTOR 

A(MID) 

A(CON) 

B 

C 


-1.84777 

0.32150 

-0.75588 

-0.17464 


OVERALL 


-0.11841 



* SIGNIFICANT AT ALPHA = .05 
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APPENDIX F-1 



AQCESS SILAS B. HAYS ARMY HOSPITAL FT ORD, CA VS AQCESS NAVAL HOSPITAL 
PENSACOLA, FL VS CHCS NAVAL HOSPITAL CHARLESTON, SC 



TOTAL SATISFACTION QUESTION INDEX 



PENSACOLA 




0.71132 


9.02586 






FT ORD 




0.80567 


10.47368 






CHARLESTON 




0.86558 


11.25248 






PENSACOLA 1 


STDEV 


1.78610 


14.93465 






FT ORD STDEV 


1.75489 


15.73504 






CHARLESTON 


STDEV 


1.65718 


11.73883 






GROUP FACTORS 


A 


A 


B 


C 








(MID) 


(CON) 








PENSACOLA 




0.89719 


0.05452 


0.69885 


0.30582 




FT ORD 




1.28363 


-0.03955 


0.94386 


0.36403 




CHARLESTON 




1.77288 


-0.35891 


0.77607 


0.52847 




PENSACOLA STDEV 


1.51598 


1.72164 


1.76039 


1.86665 




FT ORD STDEV 


1.13162 


1.77987 


1.65140 


1.77555 




CHARLESTON 


STDEV 


1.17591 


1.75196 


1.51405 


1.80255 




INDIVIDUAL 


QUESTION RESPONSES 








QUESTION 




1 


2 


3 


4 


5 


PENSACOLA 




1.49713 


-1.04011 


0.02012 


0.63793 


0.60632 


FT ORD 




1.23684 


-0.30702 


0.14035 


0.92998 


0.69299 


CHARLESTON 




1.67822 


-0.50000 


0.57426 


0.97525 


1.06436 


STAN DEV 




1 


2 


3 


4 


5 


PENSACOLA 




1.51903 


1.85706 


1.83113 


1.86497 


1.81078 


FT ORD 




1.71834 


1.79242 


1.71616 


1.54306 


1.77273 


CHARLESTON 




1.26586 


1.75133 


1.74690 


1.67640 


1.60559 


QUESTION 




6 


7 


8 


9 


10 


PENSACOLA 




1.51724 


0.75000 


0.90210 


0.64655 


0.42816 


FT ORD 




1.15789 


0.96491 


1.21930 


0.88596 


0.80702 


CHARLESTON 




1.98515 


0.79208 


0.94060 


1.16832 


0.54455 


STAN DEV 




6 


7 


8 


9 


10 


PENSACOLA 




1.56735 


1.78052 


1.69403 


1.75474 


1.71304 


FT ORD 




1.79951 


1.66980 


1.63170 


1.70530 


1.56652 


CHARLESTON 




1.04371 


1.49837 


1.43046 


1.55541 


1.18394 


QUESTION 




11 


12 


13 






PENSACOLA 




1.36494 


0.14943 


0.76724 






FT ORD 




1.45614 


0.44444 


0.84211 






CHARLESTON 




1.65347 


-0.21287 


0.59406 
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STAN DEV 
PENSACOLA 
FT ORD 
CHARLESTON 



11 12 
1.45492 1.57589 

1.66038 1.65018 

1.18394 1.74471 



13 

1.82113 

1.70951 

1.48745 
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APPENDIX F-2 



CHCS VS ACCESS 



TOTAL SATISFACTION 


CUESTION 


INDEX 




CHCS 


0.86558 


11.25248 




ACCESS 


0.73459 


10.13203 




CHCS STDEV 


1.65718 


11.73883 




ACCESS 


1.77895 


16.68588 




GROUP FACTORS 


A 


A 


B 


C 




(MID) 


(CON) 






CHCS 


1.77288 - 


-0.35891 


0.77607 


0.52847 


ACCESS 


0.89898 


0.05844 


0.75931 


0.32035 


CHCS STDEV 


1.17591 


1.75196 


1.51405 


1.80255 


ACCESS STDEV 


1.57377 


1.73917 


1.74030 


1.84525 


INDIVIDUAL CUESTION 


RESPONSES 








CUESTION 


1 


2 


3 


4 


CHCS 


1.67822 


-0. 50000 


0.57426 


0.97525 


ACCESS 


1.43290 


-0.10606 


0.04978 


0.70996 


STAN DEV 


1 


2 


3 


4 


CHCS 


1.26586 


1.75133 


1.74690 


1.67640 


ACCESS 


1.57457 


1.84215 


1.80419 


1.71322 


CUESTION 


6 


7 


8 


9 


CHCS 


1.98515 


0.79208 


0.94060 


1.16832 


ACCESS 


1.42857 


0.80303 


0.98052 


0.70563 


STAN DEV 


6 


7 


8 


9 


CHCS 


1.04371 


1.49837 


1.43046 


1.55541 


ACCESS 


1.63507 


1.75630 


1.68442 


1.74572 


CUESTION 


11 


12 


13 




CHCS 


1.65347 


-0.21287 


0.59406 




ACCESS 


1.38745 


0.22294 


0.78571 




STAN DEV 


11 


12 


13 




CHCS 


1.18394 


1.74471 


1.48745 




ACCESS 


1.50874 


1.61301 


1.79452 





5 

1.06436 

0.62771 

5 

1.60559 

1.80185 

10 

0.54455 

0.52165 

10 

1 . 18394 
1.68600 
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APPENDIX F-3 



PENSACOLA vs CHARLESTON 



CALCULATED 


T -VALUE 


1 


1.0038 


2 


2.37125 * 


3 


2.44113 * 


4 


1.51899 


5 


2.06814 * 


6 


2.68497 * 


7 


0.237516 


8 


1.99893 


9 


2.47011 * 


10 


0.57204 


11 


1.70795 


12 


-1.76089 



13 -0.843187 



FACTOR 

A(MID) -5.044247 * 
A(CON) -1.89340 
B 0.382317 

C 0.953397 



OVERALL 



1.28452 



* SIGNIFICANT AT ALPHA = .05 
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APPENDIX F-4 



FORT ORD vs CHARLESTON 



CALCULATED T-VALUE 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 
13 



-1.83458 
0.648655 
-1.49233 
-0.148027 
-1.33802 
-3.63284 * 
0.656472 
1.12314 
-1.03273 
1.22248 
-0.88007 
2.29686 * 
0.95965 



FACTOR 

A(MID) -2.55907 * 
A(CON) 1.09699 
B 0.61843 

C -0.572368 



OVERALL 



-0.353725 



* SIGNIFICANT AT ALPHA = .05 
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APPENDIX G-1 



COMBINED RESULTS FOR CHCS AND AQCESS 



GENERAL 


AVERAGE 


INDEX 






PHYSICIANS 


0.10376 


1.348837 






STDEV 


1.61997 


11.15824 






ANCILLARY 


0.86593 


11.42857 






STDEV 


1.73685 


13.86241 






ADMINISTRATIVE 


0.88229 


11.4698 






STDEV 


1.73962 


14.25219 






GROUP FACTORS 


A 


A 


B 


C 




(MID) 


(CON) 






PHYSICIAN SAT 


0.56202 


-0.46899 


0.17907 


-0.48547 


ANCILLARY SAT 


1 . 64286 


-0.08571 


0.793571 


0.559821 


ADMIN SAT 


1.55593 


0.139262 


0.927517 


0.46896 


PHYSICIAN STDEV 


1.37496 


1.360055 


1.505966 


1.640625 


ANCILLARY STDEV 


1.37705 


1.79179 


1.683394 


1.857416 


ADMIN STDEV 


1.55679 


1.768214 


1.673756 


1.796915 


OVERALL AVG 


1.55063 


-0.06777 


0.763441 


0.412634 


OVERALL STDEV 


1.45332 


1.751599 


1.66848 


1.837235 


INDIVIDUAL QUESTION 


RESPONSES 






QUESTION 


1 


2 


3 


4 


PHYSICIAN 


1 


-0.72093 


-0.27692 


0.06153 


ANCILLARY 


1.63571 


-0.29286 


0.467857 


1.02857 


ADMIN 


1.53356 


-0.01007 


0.167785 


0.88590 1 


STAN DEV 


1 


2 


3 


4 


PHYSICIAN 


1.48637 


1.413688 


1.476515 


1.75381 


ANCILLARY 


1.41275 


1.882234 


1.864816 


1.69639 


ADMIN 


1.53282 


1.834576 


1.770006 


1.70876 


QUESTION 


6 


7 


8 


9 


PHYSICIAN 


1.02326 


-0.75581 


-0.05814 


0.4418 1 


ANCILLARY 


1.73571 


0.85 


1.017857 


0.92143 1 


ADMIN 


1.63423 


0.922819 


1.218121 


0.89261 1 


STAN DEV 


6 


7 


8 


9 


PHYSICIAN 


1.46253 


1.630128 


1.630439 


1.63971 


ANCILLARY 


1.36073 


1.666583 


1.557139 


1.71575 


ADMIN 


1.59816 


1.700005 


1.56173 


1.70919 


QUESTION 


11 


12 


13 




PHYSICIAN 


1.37952 


-0.36145 


0.295181 




ANCILLARY 


1.55714 


0.121429 


0.717857 




ADMIN 


1.5 


0.288591 


0.932886 





5 

0.092 

.0357 

.8322 

5 

.8437 

.6493 

.6945 

10 

.2409 

.4607 

.6711 

10 

.4775 

.6664 

.6423 
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STAN DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 



11 12 13 

1.23476 1.517594 1.485951 
1.35104 1.671047 1.749194 
1.53538 1.686087 1.749194 
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APPENDIX G-2 



COMBINED RESULTS FOR CHCS AND AQCESS 
PHYSICIANS VS THE REST OF THE SURVEY GROUP 



GENERAL 


AVERAGE 


INDEX 








PHYSICIANS 


0.10376 


1.348837 








STDEV 


1.48637 


11.15824 








NON-PHYSICIANS 


0.87437 


11.44983 








STDEV 


1.73957 


14.06473 








GROUP FACTORS 


A 


A 


B 


C 






(MID) 


(CON) 








PHYSICIAN SAT 


0.56202 


-0.46899 


0.17907 


-0.48547 




PHYSICIAN STDEV 


1.37496 


1.360055 


1.505966 


1.640625 




NON -PHYSICIANS 


1.59804 


0.295848 


0.86263 


0.512976 




NON-PHYSICIANS STDEV 


1.4731 


1.783479 


1.679766 


1.827038 




QUESTION 


1 


2 


3 


4 


5 


PHYSICIAN 


1 


-0.72093 


-0.27692 


0.061538 


-0.09231 


NON-PHYSICIANS 


1.58304 


-0.27682 


0.313149 


0.955017 


0.930796 


STAN DEV 


1 


2 


3 


4 


5 


PHYSICIAN 


1.48637 


1.413688 


1.476515 


1.753811 


1.843711 


NON- PHYSICIANS 


1.47676 


1.863184 


1.822732 


1.704274 


1.675816 


QUESTION 


6 


7 


8 


9 


10 


PHYSICIAN 


1.02326 


-0.75581 


-0.05814 


0.44186 


0.240964 


NON-PHYSICIANS 


1.68339 


-0.14706 


1.121107 


0.906574 


0.569204 


STAN DEV 


6 


7 


8 


9 


10 


PHYSICIAN 


1.46253 


1.630128 


1.630439 


1.639705 


1.477464 


NON- PHYSICIANS 


1.48875 


1.68429 


1.562716 


1.712439 


1.65739 


QUESTION 


11 


12 


13 






PHYSICIAN 


1.37952 


-0.36145 


0.295181 






NON-PHYSICIANS 


1.52768 


0.205882 


0.82872 






STAN DEV 


11 


12 


13 






PHYSICIAN 


1.23476 


1.517594 


1.485951 






NON-PHYSICIANS 


1.44929 


1.681623 


1.730074 
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APPENDIX G-3 



PHYSICIANS vs THE REST OF THE SURVEY GROUP 
CALCULATED T-VALUE 



1 


2.40721* 


2 


1.52642 


3 


1.98308* 


4 


3.23135* 


5 


3.65143* 


6 


2.78686* 


7 


2.20323* 


8 


4.62484* 


9 


1.68446 


10 


1.19979 


11 


0.64683 


12 


2.10863* 


13 


1.91295 


FACTOR 


A(MID) 


4.38365* 


A(CON) 


2.69228* 


B 


2.49046* 


C 


3.41617* 



OVERALL 4.52104* 

* SIGNIFICANT AT ALPHA = .05 
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APPENDIX G-4 



COMBINED RESULTS FOR CHCS AND AQCESS 



ANCILLARY VS THE REST OF THE 


SURVEY GROUP 




GENERAL 


AVERAGE 


INDEX 






ANCILLARY 


0.86621 


11.42857 






ANCILLARY STDEV 


1.73663 


13.86241 






NON-ANCILLARY 


0.70753 


9.203125 






NON- ANCILLARY STDEV 


1.74605 


14.25906 






GROUP FACTORS 


A 


A 


B 


C 




(MID) 


(CON) 






ANCILLARY SAT 


1.64286 


-0.08571 


0.793571 


0.560714 


ANCILLARY STDEV 


1.37705 


1.79179 


1.683394 


1.856906 


NON-ANCILLARY 


1.43837 


-0.05599 


0.727909 


0.254557 


NON-ANCILLARY STDEV 


1.53366 


1.72398 


1.667132 


1.807729 



QUESTION 


1 


2 


3 


4 


5 


ANCILLARY 


1.63571 


-0.29286 


0.467857 


1.028571 


1.039286 


NON-ANCILLARY 


1.41406 


-0.17708 


0.020833 


0.617188 


0.557292 


STAN DEV 


1 


2 


3 


4 


5 


ANCILLARY 


1.41275 


1.882234 


1.864816 


1.696395 


1.64595 


NON-ANCILLARY 


1.53871 


1.776553 


1.730421 


1.790256 


1.80331 


QUESTION 


6 


7 


8 


9 


10 


ANCILLARY 


1.73571 


0.85 


1.017857 


0.921429 


0.460714 


NON-ANCILLARY 


1.4974 


0.763021 


0.932292 


0.791667 


0.578125 


STAN DEV 


6 


7 


8 


9 


10 


ANCILLARY 


1.36073 


1.666583 


1.557139 


1.715758 


1.666442 


NON-ANCILLARY 


1.58935 


1.746181 


1.647472 


1.690455 


1.608616 


QUESTION 


11 


12 


13 






ANCILLARY 


1.55714 


0.121429 


0.717857 






NON-ANCILLARY 


1.40365 


0.065104 


0.734375 






STAN DEV 


11 


12 


13 






ANCILLARY 


1.35104 


1.671047 


1.749194 






NON-ANCILLARY 


1.4688 


1.660958 


1.677754 
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APPENDIX G-5 



ANCILLARY vs THE REST OF THE SURVEY GROUP 



CALCULATED 


T-VALUE 


1 


-1.45036 


2 


0.61534 


3 


-2.40663* 


4 


-2.18425* 


5 


-2.55840* 


6 


-1.44183 


7 


-0.49104 


8 


-0.46473 


9 


-0.71419 


10 


0.68605 


11 


-1.06003 


12 


-0.30922 


13 


0.10936 


FACTOR 


A(MID) 


-1.28482 


A(CON) 


0.11483 


B 


-0.33643 


C 


-1.56279 


OVERALL 


-1. 


* SIGNIFICANT AT ALF 
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APPENDIX G-6 



COMBINED RESULTS FOR CHCS AND AQCESS 
ADMINISTRATIVE VS THE REST OF THE SURVEY GROUP 



GENERAL 




AVERAGE 


INDEX 








ADMINISTRATIVE 


0.88178 


11.4698 








ADMIN STDEV 


1.74002 


14.2522 








NON-ADMIN 




0.68705 


9.06011 






NON-ADMIN 


STDEV 


1.74022 


14.25906 






GROUP FACTORS 


A 


A 


B 


C 








(MID) 


(CON) 








ADMIN SAT 




1.55593 


0.137584 


0.927517 


0.468121 




ADMIN STDEV 


1.55679 


1.768819 


1.673756 


1.797367 




NON -ADM IN 


SAT 


1.52302 


-0.23634 


0.64918 


0.314891 




NON-ADMIN 


STDEV 


1.40079 


1.721812 


1.66395 


1.862015 




QUESTION 




1 


2 


3 


4 


5 


ADMIN 




1.53356 


-0.01007 


0.167785 


0.885906 


0.828859 


NON-ADMIN 




1.48634 


-0.40164 


0.243169 


0.713115 


0.704918 


STAN DEV 




1 


2 


3 


4 


5 


ADMIN 




1.53282 


1.834576 


1.770006 


1.708762 


1.697088 


NON -ADM IN 




1.45556 


1.794009 


1.826758 


1.802304 


1.798703 


QUESTION 




6 


7 


8 


9 


10 


ADMIN 




1.63423 


0.922819 


1.218121 


0.892617 


0.671141 


NON-ADMIN 




1.56831 


0.699454 


0.765027 


0.808743 


0.412568 


STAN DEV 




6 


7 


8 


9 


10 


ADMIN 




1.59816 


1.700005 


1.56173 


1.709197 


1.642322 


NON- ADMIN 




1.41787 


1.658675 


1.621064 


1.695874 


1.618458 


QUESTION 




11 


12 


13 






ADMIN 




1.5 


0.285235 


0.932886 






NON-ADMIN 




1.44262 


-0.07104 


0.560109 






STAN DEV 




11 


12 


13 






ADMIN 




1.53538 


1.687653 


1.749194 






NON- ADMIN 




1.32266 


1.629772 


1.692298 







172 



APPENDIX G-7 



ADMINISTRATIVE 


VS THE REST 


CALCULATED T-VALUE 


1 


0.00000 


2 


-2.01622 * 


3 


-0.36259 


4 


-0.95500 


5 


-0.69139 


6 


-0.43885 


7 


-1.09948 


8 


-2.65698 


9 


-0.52758 


10 


-1.57692 


11 


-0.39612 


12 


-1.97450 * 


13 


-2.00788 * 


FACTOR 


A(MID) 


-0.25394 


A(CON) 


-1.82145 


B 


-1.68437 


C 


-1.01902 


OVERALL 


-1.596 


* SIGNIFICANT AT ALPHA 
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